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Abstract 

Nonlinear  regulation  and  nonlinear  Hqo  control  via  state-dependent  Riccati  equation  (SDRE) 
techniques  are  considered.  Relationships  between  Hamilton- Jacobi/Bellman  inequalities/ equations 
and  SDREs  are  examined,  and  a  necessary  condition  for  existence  of  solutions  involving  nonlinear 
stabilizability  is  derived.  A  single  additional  necessary  criterion  is  given  for  the  SDRE  methods 
to  yield  the  optimal  control  or  guaranteed  induced  L2  gain  properties.  Pointwise  stabilizability 
and  detectability  of  factorizations  prove  necessary  and  sufficient,  respectively,  for  well-posedness  of 
standard  numerical  implementations  of  suboptimal  SDRE  regulators,  but  neither  proves  necessary 
if  analytical  solutions  are  allowed.  For  scalar  analytic  systems  or  those  with  full  rank  constant 
control  input  matrices,  stabilizability  and  nonsingularity  of  the  state  weighting  matrix  function 
result  in  local  and  global  asymptotic  stability,  respectively,  due  to  equivalence  between  nonlinear 
and  factored  controllability  in  these  cases.  A  proof  of  asymptotic  stability  for  sampled  data  analytic 
SDRE  controllers  is  also  given,  but  restrictive  assumptions  make  the  main  utility  of  these  results 
guidance  in  choosing  appropriate  system  factorizations.  Conditions  for  exponential  stability  are  also 
derived.  All  results  are  extendable  to  SDRE  nonlinear  Hoo  control  with  additional  assumptions. 
The  SDRE  theory  is  illustrated  by  application  to  momentum  control  of  a  dual-spin  satellite  and 
comparison  with  other  current  methods. 
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L  Introduction 


1.1  Background 

Over  the  past  several  years,  the  relative  maturity  of  linear  controller  design  techniques,  coupled 
with  the  inability  of  these  techniques  to  handle  strongly  nonlinear  dynamic  systems  satisfactorily, 
has  led  to  a  push  in  the  development  of  nonlinear  controller  synthesis  theory  and  techniques.  As 
a  result,  theory  has  emerged  for  design  according  to  a  number  of  methods,  including  feedback 
linearization  [33,  52,  62],  variable  structure  control  [21,  62],  control  Lyapunov  functions  [2,  64], 
recursive  backstepping  [40]  and  nonlinear  Hqo  control  [3,  34,  66].  Also,  a  state-dependent  Riccati 
equation  (SDRE)  technique  for  nonlinear  control  has  recently  appeared  in  the  literature  [13,  14,  55]. 
Each  method  is  currently  at  a  different  state  of  maturity,  and  each  contains  inherent  limitations 
which  restrict  applicability  to  any  given  problem.  A  brief  discussion  of  each  method  follows. 

Feedback  linearization  is  one  of  the  more  mature  nonlinear  controller  synthesis  techniques,  having 
its  basis  in  differential  geometry,  and  receiving  comprehensive  treatment  in  several  texts  [33,  52,  62]. 
The  basic  idea  is  to  find  a  coordinate  transformation  and  static  feedback  which  renders  the  closed  loop 
system  linear  and  stabilizable,  and  then  to  employ  the  wealth  of  existing  linear  controller  synthesis 
techniques  to  the  transformed  system  to  achieve  desired  performance  objectives.  Theory  exists 
primarily  for  state  feedback  but  also  for  output  feedback  cases,  and  formulas  for  analytical  solutions 
to  stability,  tracking,  disturbance  decoupling,  and  noninteracting  control  problems  exist  which  may 
be  solved,  provided  the  system  under  consideration  is  feedback  linearizable.  The  difficulties  with  this 
method  stem  from  the  fact  that  a  given  input-output  system  either  possesses  the  desired  feedback 
linearizability  property  or  it  does  not.  If  the  system  is  completely  linearizable  (all  of  the  state 
equations  may  be  made  linear),  then  design  may  proceed.  If  the  system  is  not  completely  linearizable 
and  if  the  nonlinearized  (unobservable)  zero  dynamics  are  asymptotically  stable,  controller  design 
may  proceed  on  the  linearized  portion  of  the  state.  However,  if  the  zero  dynamics  are  unstable,  this 
method  may  not,  in  general,  be  used.  Tests  exist  [33,  62]  for  checking  the  feedback  linearizability 
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of  a  system  and  the  stability  of  the  resulting  zero  dynamics,  and  if  a  system  is  found  not  to  pass 
the  tests,  this  method  is  not  applicable.  An  additional  disadvantage  of  this  method  is  that  it 
completely  removes  all  nonlinearities  from  a  system,  whether  they  are  beneficial  (e.g.,  with  regard 
to  stability)  or  not.  Thus,  the  method  is  nonoptimal.  Finally,  technically  only  a  number  of  outputs 
equal  to  the  number  of  inputs  can  be  controlled,  posing  an  additional  design  constraint.  For  cases 
where  there  are  more  inputs  than  outputs,  pseudo-controls  can  be  formed.  Several  examples  of 
application  of  this  method  to  suitable  systems  can  be  found  in  the  literature  [43,  61,  63],  and  current 
research  in  this  area  focuses  on  extending  the  applicability  of  the  method  to  systems  which  currently 
may  not  be  addressed  (systems  not  completely  linearizable  or  with  unstable  zero  dynamics).  Also, 
extending  other  differential  geometry  concepts  to  achieve  more  advanced  performance  objectives  is 
being  pursued. 

Variable  structure  control  (VSC),  also  known  as  sliding  mode  control  [62],  is  another  relatively 
mature  nonlinear  controller  synthesis  method.  The  main  objective  is  to  define  a  manifold  in  which 
it  is  desired  to  keep  the  system  state,  and  then  to  drive  the  system  to  this  manifold  (i.e.,  make  the 
manifold  attractive  in  the  closed  loop).  Variable  structure  control  results  in  discontinuous,  high-gain 
controllers,  with  ensuing  chattering  of  the  control.  The  main  advantage  of  VSC  is  that  its  high-gain 
nature  yields  desirable  robustness  properties  under  certain  conditions,  making  VSC  a  good  candidate 
for  control  of  systems  with  appropriately  modeled  uncertainty  [62].  Current  research  is  attempting 
to  eliminate  the  chattering  of  the  controls,  and  seeks  optimal  definitions  for  the  desired  state  space 
manifold. 

The  idea  behind  control  Lyapunov  functions  is  to  try  to  find  a  Lyapunov  function  [68]  for  the 
closed  loop  system  and  to  derive  a  controller  which  guarantees  stability,  based  on  application  of  the 
Direct  Lyapunov  Method  [36]  to  the  closed  loop  system  equations.  Necessary  and  sufficient  condi¬ 
tions  have  been  derived  for  existence  of  control  Lyapunov  functions  for  certain  classes  of  problems 
[2],  but  as  yet  no  systematic  method  for  finding  them  has  been  proposed.  Thus,  this  method  is 
immature  in  terms  of  synthesis  capability. 
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Recursive  backstepping  is  another  relatively  new  technique  being  advocated  as  an  extremely 
flexible  yet  systematic  methodology  for  direct  nonlinear  control  design.  A  recent  text  gives  a  com¬ 
prehensive  treatment  of  the  method  [40],  claiming  that  its  primary  advantage  lies  in  its  ability  to 
address  adaptive  control  of  nonlinear  systems,  thus  going  one  step  beyond  simple  nominal  control 
design.  The  basic  idea  behind  this  method  is  sequential  scalar  control  design  for  each  nonlinear 
state  differential  equation,  incorporating  the  use  of  subsequent  states  or  combinations  of  states  as 
pseudocontrols  in  the  previous  designs.  The  actual  physical  control  is  made  to  appear  in  the  final 
state  equation,  and  thus  is  the  last  quantity  chosen  to  attempt  to  meet  the  overall  design  objec¬ 
tives.  The  method  is  Lyapunov  function  based,  and  thus  aims  to  guarantee  closed  loop  stability. 
The  designer  attempts  to  obtain  other  performance  objectives  by  “optimizing”  coefficients  in  the 
Lyapunov  function,  and  by  clever  choice  of  pseudocontrols.  Prom  this  discussion,  it  can  be  seen  that 
recursive  backstepping  is  really  an  application  and  extension  of  control  Lyapunov  function  theory 
[40].  Recursive  backstepping,  as  with  other  methods,  is  limited  in  the  types  of  systems  to  which 
it  can  be  applied,  and  although  the  general  theory  is  systematic,  the  actual  synthesis  details  are 
problem-  and  designer-specific.  Thus,  the  method  can  yield  several  different  designs  depending  on 
the  designer’s  insight  and  the  particular  choices  made. 

Nonlinear  Hoo  control  theory  is  currently  in  the  early  stages  of  development  [3,  4,  31,  34,  35, 
66,  67,  75].  The  current  state-of-the-art  consists  of  sufficient  conditions  for  obtaining  nonlinear  con¬ 
trollers  which  guarantee  induced  L2  performance  and  local  closed  loop  stability  for  both  state  and 
output  feedback  cases.  In  the  state  feedback  case,  the  sufficient  condition  corresponds  to  finding  a 
locally  positive  definite  solution  to  a  certain  Hamilton-Jacobi  inequality  (HJI)  or  Hamilton-Jacobi- 
Issacs  equation  (HJIE)  (or  finding  a  positive  semidefinite  solution  with  additional  assumptions  re¬ 
quired),  which  depends  parametrically  on  the  induced  L2  gain.  Solving  the  output  feedback  problem 
requires  solution  of  another  HJI  and  ensuring  satisfaction  of  a  third  condition  relating  the  solutions 
of  the  two  HJIs.  Nontrivial  solutions  have  been  found  for  limited  special  cases,  mainly  involving 
lossless/dissipative  systems  and  state  feedback  [15,  16,  37,  48,  71].  For  the  general  case,  however, 
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although  initial  (numerical)  solution  attempts  have  been  made  [29],  little  information  exists  in  the 
literature  describing  how  solutions  to  HJIs  can  be  found,  let  alone  guaranteeing  satisfaction  of  the 
HJI  solution  coupling  condition  of  the  output  feedback  problem.  Also,  it  remains  to  discover  a  way 
to  compute  the  optimal  induced  L2  gain,  although  recently  methods  for  its  approximation  have  ap¬ 
peared  in  the  literature  [35,  75].  Thus,  much  work  remains  to  be  done  in  characterizing  and  actually 
obtaining  solutions  to  the  nonlinear  Hqo  synthesis  problem. 

Finally,  a  state- dependent  Riccati  equation  (SDRE)  based  technique  for  nonlinear  regulation, 
originally  proposed  as  far  back  as  1962  [54],  has  recently  seen  renewed  interest  [13,  14,  55].  This 
method  has  been  described  as  a  nonlinear  extension  of  the  well  known  two-Riccati-equation  solution 
technique  to  linear  quadratic  Gaussian  (LQG)  type  synthesis  problems.  SDRE  solution  techniques 
for  both  state  and  output  feedback  nonlinear  Hoq  control  have  also  been  proposed,  but  not  developed. 
Even  though  variants  of  this  method  have  been  proposed  for  a  number  of  years  [9,  19,  54,  70],  the 
underlying  theory  is  scant,  despite  the  fact  that  results  from  its  application  have  been  impressive 
[14,  54].  Several  academic  problems  have  been  solved  analytically,  and  a  numerically  solved  output 
feedback  SDRE  regulator  solution  has  been  obtained  for  a  nonlinear  missile  autopilot  [12],  indicating 
the  feasibility  of  applying  this  method  to  realistic  problems.  Also,  a  thesis  at  AFIT  [53]  was  recently 
completed  applying  this  method  to  satellite  control  and  control  of  an  artificial  human  pancreas.  A 
tremendous  need  exists,  however,  to  fill  in  a  large  number  of  theoretical  ‘holes’  for  this  method. 
Stability,  optimality,  factorization/parametrization  techniques,  solution  methods  and  a  number  of 
other  issues  still  need  to  be  analyzed  in  detail. 

We  close  this  section  with  the  observation  that  the  above  discussion  attempts  to  focus  on  nonlin¬ 
ear  control  techniques  which  have  emerged  relatively  recently,  and  thus,  the  long- established  optimal 
control  techniques  based  on  variational  calculus  and/or  dynamic  programming  have  been  intention¬ 
ally  omitted.  These  are  certainly  valid  techniques  for  control  design  for  nonlinear  systems,  but  the 
theoretical  basis  for  them  is  quite  mature  and  well  documented.  See,  for  example  [8].  Applica¬ 
tion  of  the  theory  generally  results  in  nonlinear,  constrained,  two-point  boundary  value  problems 
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(variational  calculus)  or  partial  differential  (Bellman)  equations  (dynamic  programming),  analytical 
solutions  to  either  of  which  are  extremely  difficult  to  obtain  for  most  nontrivial  problems  of  interest. 
Although  some  research  continues  in  the  area  of  control  design  by  these  methods,  it  mainly  focuses 
on  application  of  the  relevant  theory  to  limited  special  cases  to  obtain  the  form  of  the  optimality 
equations,  or  on  numerical  attempts  to  solve  them,  and  not  on  extending  the  theory  itself.  We 
mention  them  in  passing  here  because  some  techniques  examined  in  this  document,  particularly  the 
SDRE  nonlinear  regulation  method,  may  be  viewed  as  new  ways  of  solving  special  cases  of  such 
optimal  control  problems,  and  thus  must  yield  solutions  equivalent  to  those  obtained  from  using  the 
pertinent  theory. 

L2  Research  Objectives  and  Contributions 

As  can  be  seen  from  the  preceding  discussion,  a  number  of  potential  design  techniques  now  exists  for 
direct  nonlinear  control  design.  Although  several  methods  are  fairly  well  established  theoretically, 
many  of  them  have  severe  limitations  which  restrict  their  applicability  or  usefulness  to  realistic 
control  design  problems.  This  research  focuses  on  providing  a  sound  theoretical  basis  for  control 
of  nonlinear  systems  via  the  state  feedback  SDRE  techniques,  which,  as  discussed  above,  have 
proven  quite  successful  in  a  number  of  simulated  applications,  but  lack  supporting  theory  in  a 
number  of  areas.  Many  of  these  issues  are  attacked  herein,  with  significant  headway  being  made 
in  theoretical  justification  for  design  choices  and  properties.  During  the  course  of  our  presentation, 
wherever  possible  we  seek  to  draw  comparisons  between  the  SDRE  techniques  developed  herein  and 
other  established  nonlinear  control  techniques,  particularly  the  methods  of  feedback  linearization, 
recursive  backstepping,  Hamilton- Jacobi-Bellman  optimal  control,  and  nonlinear  Hoo  control.  To 
this  end  a  summary  of  the  theory  relevant  to  these  methods  is  included  herein,  and  a  motivational 
design  example  applying  all  these  techniques  to  the  same  problem  is  pursued  in  detail.  In  addition 
to  theoretical  development,  we  also  make  a  nontrivial  application  of  the  SDRE  control  techniques 
to  a  problem  of  interest  to  the  U.S.  Air  Force.  The  particular  control  problem  addressed  is  gyrostat 
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satellite  angular  momentum  control  [22,  23,  25].  This  problem  has  characteristics  which  invite 
the  use  of  direct  nonlinear  control  techniques.  The  gyrostat  problem  exhibits  strongly  nonlinear 
dynamics:  so  much  so  that,  in  fact,  the  linearized  dynamics  for  the  baseline  problem  are  zero.  The 
lack  of  a  stabilizable  linearization  for  this  problem  poses  a  severe  challenge  to  the  control  design 
techniques,  requiring  a  configuration  modification  to  obtain  acceptable  solutions.  These  types  of 
issues  are  often  ignored  in  purely  academic  studies  of  nonlinear  control  design,  but  are  included 
herein  in  our  examination  of  applicability  of  design  methods.  A  summary  of  general  and  specific 
research  objectives  considered  is  given  below.  The  remainder  of  this  dissertation  serves  to  document 
their  satisfaction  and  illustrate  the  contributions  of  this  research. 

1.2 A  General  Research  Objectives 

•  To  investigate  the  solvability  of  state-of-the-art  control  synthesis  algorithms  for  nonlinear  dy¬ 
namic  systems  of  practical  interest  to  the  U.S.  Air  Force 

•  To  develop  methods  for  solving  the  above  synthesis  problems  more  easily 

•  To  extend  the  applicability  of  current  methods  where  possible 

•  To  make  an  objective  determination  of  the  relative  merits  and  disadvantages  of  the  various 
techniques  considered 

1.2.2  SDRE  Method-Specific  Research  Objectives 

•  To  investigate  the  relative  tradeoffs  between  analytical /numerical  solution  of  SDREs 

•  To  analyze  the  effects  of  various  state-dependent  parametrizations 

•  To  examine  correlations  with  existing  proposed  nonlinear  Hoo  solution  techniques 

•  To  identify  and  prove  conditions  leading  to  closed  loop  stability 

•  To  investigate  sampled  data  implementation  issues 
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•  To  discover  relationships  between  SDRE  and  optimal  nonlinear  regulation 

•  To  seek  correlations  with  the  known  linear  Riccati,  if2,  and  Hoo  theory 

•  To  apply  the  method  to  control  design  of  a  nontrivial  nonlinear  dynamic  system 

•  To  make  an  objective  determination  of  the  relative  merits  and  disadvantages  of  the  design 
technique 


1.3  ^Linear’  Control  of  Nonlinear  Systems  -  A  Literature  Review 

When  speaking  strictly  of  SDRE  techniques,  the  literature  citations  are  few.  However,  over  the  past 
several  years,  a  number  of  researchers  have  proposed  nonlinear  control  algorithms  which  involve 
application  of  linear  design  methods  to  linear-like  ‘factored’  representations  of  a  nonlinear  system 
[9,  13,  19,  21,  54,  55,  70].  For  continuous  time,  state  feedback,  input-affine,  autonomous  nonlinear 
dynamic  systems  of  the  form 


X  =  a(x)  H-  h{x)u,  a(0)  =  0 
h{x) 


z  — 


,  h{0)  =  0 


(1.1) 


1^  R{x)u  J 

with  state  vector  x  E  control  vector  n  E  penalized  variable  z  and  nonsingular  (for  all 

x)  control  penalty  matrix  function  R{x),  it  is  assumed  that  one  can  obtain  a  factored  representation 
of  the  form 


(1.2) 


X  ~  A{x)x  -h  B{x)u 
H{x)x 
R{x)u 

so  that  at  each  point  in  the  state  space  the  nonlinear  vector  fields  a{x)  and  h{x)  have  a  linear  appear¬ 
ance  given  by  A{x)  and  H{x).  This  concept  has  alternatively  been  called  ‘equivalent  linearization’ 
[9,  54],  ‘apparent  linearization’  [70],  ‘extended  linearization’  [21],  or  most  recently  obtaining  a  ‘state- 
dependent  coefficient  factorization’  [13]  for  (1.1).  In  [54],  a  suboptimal  approach  to  the  finite  time 
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optimal  regulator  problem  for  input- affine  nonlinear  systems  is  proposed,  which  requires  solution 
of  a  state-dependent  Riccati  differential  equation  (RDE)  involving  the  factorization  (1.2).  Several 
low-order  example  problems  are  solved  analytically,  subop timal  versus  optimal  performance  compar¬ 
isons  are  made,  and  global  asymptotic  stability  is  proven  for  the  suboptimally-controlled  systems. 
In  [9],  a  variation  of  the  above  approach  is  proposed,  in  which  the  RDE  is  solved  by  power  series 
expansion  of  the  RDE  solution  with  respect  to  the  states  about  a  user-selected  fixed  point,  solving 
a  nonstate-dependent  RDE  to  obtain  the  zeroth-order  solution  term,  and  solving  several  linear  ma¬ 
trix  differential  equations  to  obtain  the  higher-order  coefficients  in  the  power  series  expansion  for 
the  state-dependent  RDE  solution.  In  [70] ,  a  suboptimal  solution  for  the  nonlinear  state  feedback 
infinite  time  horizon  quadratic  regulation  problem  is  proposed,  which  involves  solving  an  algebraic 
state-dependent  Riccati  equation  (SDRE)  for  any  location  traversed  in  the  state  space.  In  [19], 
the  same  basic  idea  is  revisited,  and  conditions  relating  the  suboptimal  solution  to  the  optimal  so¬ 
lution  are  derived.  In  both  these  papers,  the  state  and  control  weights  R  and  H  are  assumed  to 
be  constant  matrices,  so  that  the  regulation  problem  is  indeed  quadratic.  More  recently,  in  [21] 
application  of  any  linear  control  algorithm  to  (1.2)  is  suggested,  but  no  theoretical  justification  for 
such  an  approach  is  given.  In  [13]  both  state  and  output  feedback  SDRE  approaches  to  regulation 
and  nonlinear  Hqo  control  problems  are  proposed,  where  weighting  matrices  are  not  restricted  to  be 
constants,  but  may  instead  be  functions  of  x.  Local  stability  is  proven  for  suboptimal  state  feedback 
versions  of  the  above,  and  an  additional  necessary  condition  which  must  be  satisfied  for  optimality  of 
the  state  feedback  regulator  is  given.  In  [55]  a  Lyapunov  function  is  proposed  for  establishing  global 
stability  of  the  suboptimal  state  feedback  regulator,  based  on  a  restricted  class  of  weighting  matrix 
functions.  Finally,  in  [30],  the  SDRE  nonlinear  regulator  is  examined  as  an  alternative  to  solving 
Hamilton-Jacobi-Issacs  equations  in  the  optimal  regulation  problem,  and  sufficient  conditions  for  ex¬ 
istence  of  an  ‘optimal’  state-dependent  factorization  are  given.  While  some  progress  has  been  made 
in  theoretically  justifying  such  ‘linearized’  methods,  much  remains  to  be  done.  Important  areas 
such  as  analytical  versus  numerical  solution  techniques,  guaranteed  stability  beyond  the  domain  of 
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attraction  of  the  system  linearization,  controllability  issues,  choices  of  factorizations,  sampled  data 
applications,  and  optimality  remain  to  be  fully  investigated-  It  is  precisely  these  issues  which  are 
addressed  in  this  research. 

1.4  Overview 

The  remainder  of  this  document  is  organized  as  follows.  Chapter  2  summarizes  the  existing  theory 
relevant  to  nonlinear  stability  analysis  and  control  design  via  the  four  methods  discussed  above. 
Chapter  3  extends  the  presentation  of  material  in  Chapter  2  by  showing  application  of  each  method 
to  an  academic  second-order  design  example.  This  initial  attempt  at  design  gives  an  early  indication 
of  each  method’s  utility,  strengths,  and  weaknesses,  and  thereby  points  to  promising  specific  areas 
of  research.  The  next  several  chapters  summarize  the  theoretical  advances  achieved  by  this  research. 
Chapter  4  contains  a  number  of  insights  into  various  design  issues,  in  particular  addressing  convexity, 
optimality,  and  analytical  versus  numerical  solution  approach  tradeoffs.  In  Chapter  5,  we  explore 
the  properties  of  scalar  analytic  systems  regulated  by  SDRE  control,  developing  necessary  and 
sufficient  conditions  for  existence  of  locally  stabilizing  analytic  solutions.  In  Chapter  6,  we  explore 
the  correlations  between  true  nonlinear  system  controllability  and  controllability  of  state-dependent 
factorizations,  and  explain  the  impact  of  each  on  system  stability.  We  build  on  a  special  case  of  these 
results  in  Chapter  7,  in  which  we  prove  global  asymptotic  stability  of  continuous  time  systems  with 
full  rank,  constant  B  matrices  and  globally  positive  definite  state  weighting  matrix  functions.  In  the 
next  two  chapters  we  analyze  sampled  data  implementations  of  SDRE  nonlinear  regulators,  deriving 
sufficient  conditions  for  semiglobal  asymptotic  stability  of  the  closed  loop  system.  Then,  in  Chapter 
10,  we  explore  some  issues  relating  to  exponential  stability  of  SDRE  regulators.  In  Chapter  11,  we 
revisit  the  theory  of  Chapters  5-10,  showing  that  it  may  be  extended  to  the  setting  of  nonlinear 
Hoo  control  via  the  SDRE  method,  provided  some  suitable  additional  assumptions  are  made.  In 
Chapter  12,  the  proposed  satellite  control  design  problems  and  dynamic  models  are  presented,  and 
results  of  design  iterations  and  simulations  are  shown,  illustrating  the  theory  laid  out  in  the  previous 


1-9 


chapters.  Finally,  Chapter  13  concludes  this  dissertation  by  summarizing  the  conducted  research 
and  highlighting  specific  areas  of  contribution. 
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II.  Background  Theory 


This  chapter  presents  background  theory  necessary  for  stability  analysis  and  synthesis  of  nonlinear 
dynamic  control  systems.  In  Section  2.1,  relevant  analysis  tools  are  presented,  consisting  primarily 
of  fundamental  stability  concepts  and  theory.  The  remaining  sections  of  Chapter  2  cover  the  basic 
requisite  theory  for  direct  synthesis  of  nonlinear  control  systems  for  the  methods  mentioned  in 
Section  1.2,  namely  feedback  linearization,'  recursive  backstepping,  nonlinear  control,  and  the 
SDRE  methods,  respectively. 

2.1  Stability  Analysis  of  Nonlinear  Dynamic  Systems 
2,1.1  Lyapunov  Based  Concepts 

We  now  address  fundamental  terminology  and  theory  needed  for  discussing  stability  of  nonlinear  dy¬ 
namic  systems.  More  precisely,  the  following  discussion  refers  to  solutions  and  equilibria  of  dynamic 
systems  and  not  to  the  systems  themselves.  The  following  definitions  are  taken  primarily  from  [40]. 
Let  7?.-)-  be  the  set  of  nonnegative  real  numbers,  and  consider  a  nonlinear,  nonautonomous  dynamic 
system  with  equations  of  motion  of  the  form 

X  ~  a{x^  t)  (2.1) 

with  state  x  6  7^”^,  and  a  :  'RT'  x  7?.+  — TZ^  piecewise  continuous  in  t  and  locally  Lipschitz  in  x. 
The  solution  of  (2.1)  which  starts  from  the  point  Xq  at  time  >  0  is  denoted  as  x{t\Xo-^tQ)  so  that 
x{tQ;xQ,tQ)  =  xq.  Let  ^0  ^  be  an  equilibrium  of  (2.1),  i.e.,  a{xQ,t)  =  0  V  i  G  TZ.\.,  and  let  ||a;|| 
represent  the  Euclidean  norm  of  a;  G  7^”^.  The  following  definitions  will  prove  useful.  We  shall  say 
that  the  (unique)  solution  x{t\XQ^tQ)  of  (2.1)  is 

•  bounded,  if  3  a  constant  B{xQ^tC)  >  0  such  that 

\\x{t]XQ,to)\\  <  B{xQ,tQ)  \/  t>tQ  (2.2) 
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•  (Lyapunov)  stable,  if  for  each  £:  >  0  3  a  S{s^  to)  >  0  such  that 

po  -^’oll  <  ^  ^  \\x{t]Xo,to)  ”  x{t;xQ,to)\\  <  e'i  t>  to 

•  attractive,  if  3  an  r(io)  >  0  and  for  each  £  >  0  3  a  r(£,  to)  >  0  such  that 

ll^’o  -a?o||  <  ||a;(t;:ro,to)  -  x{t;xo,to)\\  <£:Vt>to+T  (2.4) 

•  asymptotically  stable,  if  it  is  both  stable  and  attractive,  so  that  if  we  define  ex{t)  = 
\\x{t;xo,to)  -  x{t]Xo,to)\\,  the  solution  x{t]Xo,to)  is  said  to  be  asymptotically  stable  if 

lim  ex{t)  =  0  (2.5) 

t-^oo 

•  exponentially  stable,  if  asymptotic  stability  holds  with  an  exponential  upper  bound  on  the 
rate  of  convergence,  so  that 

exit)  <  ke~'^^ex{0)  (2.6) 

for  some  k  >  0,  a  >  Q. 

•  unstable,  if  it  is  not  stable. 

If  the  stability  properties  of  a  solution  are  independent  of  to,  they  are  said  to  be  uniform.  Thus, 
all  time-invariant  systems  have  uniform  stability  properties.  All  of  the  above  terms  except  bounded 
can  also  be  used  to  describe  equilibria  of  (2.1).  A  stable  or  asymptotically  stable  equilibrium, 
has  a  region  of  attraction  -  a  set  n  of  initial  states  Xq  which  either  remain  close  to  iii  Ibe 
sense  of  (2.3)  (for  stable),  or  converge  to  xq  as  in  (2.5)  (for  asymptotically  stable).  If  H  =  7^^, 
the  stability  properties  of  xq  are  said  to  be  global.  Note  that  the  definitions  of  (2.3)  and  (2.4) 
require  the  existence  of  an  open  ball  in  Bn,  centered  at  and  with  radius  S  or  r,  respectively 
{Bnixo,  0  ^  {x  elZ^  \  \\x  -  Xo\\  <  ^}),  such  that  Bn  Cfl.  Thus,  a  region  of  attraction  extends  at 
least  some  finite  distance  in  all  directions  from  a  stable  equilibrium. 

As  a  matter  of  convenience,  if  we  are  interested  in  examining  the  stability  of  an  equilibrium 
point,  we  will  often  translate  coordinates  by  the  relation  z  ==  x  —  xo,  that  in  the  new  coordinate 
system  we  have 
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z  —  X  ~  a{z  -1“  xo,  t)  =  a{z^  t) 


(2.7) 


implying 

a(0,  t)  =  a{xQ,  t)  =  0  (2.8) 

Thus,  in  the  new  coordinate  system,  =  0  is  an  equilibrium  of  (2.7).  Without  loss  of  generality  we 
will  therefore  take  the  equilibrium  in  (2.1)  to  be  the  origin  in  what  follows. 

We  are  interested  in  ways  of  determining  the  stability  of  an  equilibrium,  without  solving  (2.1) 
directly.  As  a  preliminary,  we  need  a  few  more  definitions.  A  function  V  :  C7  where  U  is 

some  neighborhood  of  the  origin  (an  open  set  containing  x  =  0),  is  said  to  be 

•  positive  definite  (>  0)  on  C/  iff  1^(0)  =  0  and  V{x)  >  0\/  x  e  U,  x  ^0 

•  positive  semidefinite  (>  0)  on  C/  iff  V(0)  ==  0  and  V{x)  >0^/  x  E  U 

•  negative  definite  (<  0)  on  iff  F  (0)  =  0  and  V{x)  <  OV  x  eU,  x  ^  0 

•  negative  semidefinite  (<  0)  on  {/  iff  y(0)  =  0  and  V{x)  <0\/  x  eU 

•  radially  unbounded  ifU  =  and  y(a;)  oo  as  ||x||  ^  oo 

Definiteness  of  a  time- varying  function,  V{x^  t),  is  defined  by  requiring  the  existence  of  a  continuous, 
nondecreasing,  and  definite  function,  W{x),  as  defined  above  which  bounds  V{x^t)  away  from  zero 
in  the  appropriate  way.  For  example,  for  V{Xj  t)  to  be  positive  definite  requires 

W{0)  =  0,  V{x,  t)  >  W(x)  >0VxEU,x^O,\/t>0 

Another  useful  concept  in  Lyapunov  stability  theory  for  time- varying  functions  is  that  of  a  function 
V {x,  t)  being  decrescent,  which  implies  the  existence  of  a  continuous,  nondecreasing  scalar  function, 
Z,  such  that  Z(0)  —  0  and,  for  all  i  >  0, 

V{x,t)<Zi\\x\\) 

We  now  present  theorems  which  accomplish  the  above-stated  objective. 
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Theorem  2.1.1  (Lyapunov)  Let  x  ^0  be  an  equilibrium  of  (2,1)  and  let  i7(0,  r)  G  TZ^  be  an  open 
ball  of  radius  r  G  7^+  centered  at  the  origin  as  defined  above.  Also  letV  :U  be  a  continuously 

differentiable  positive  definite  function  V (x)  on  U  such  that 

V  =  ^^a{x,  t)  <  0Vi>  Oi'i  X  eU  (2.9) 

ax 

Then  the  origin  is  a  locally  stable  equilibrium  of  (2,1).  IfV  in  (2.9)  is  strictly  less  than  zero^  then 
the  origin  is  a  locally  asymptotically  stable  (LAS)  equilibrium  of  (2,1). 


Proof:  See  [68],  Theorems  8.1,  8.2.  ■ 

A  function  V  satisfying  all  the  conditions  given  in  Theorem  2.1.1  is  often  called  a  Lyapunov 
function  for  (2.1).  The  above  basic  stability  theorem  only  provides  local  results.  The  following 
theorem  allows  us  to  obtain  global  stability  results. 

Theorem  2.1.2  (Lasalle-Yoshizawa)  Let  x  —  Q  be  an  equilibrium  of  (2,1)  and  let  V  :  TZ^  7^+ 
be  a  continuously  differentiable^  positive  definite^  and  radially  unbounded  function  V (x)  such  that 

V  =  ^a(x,  t)  <  -W{x)  <0\/  t>0,y  X  elZ"-  (2.10) 

OX 

where  W  is  a  continuous  function.  Then,  all  solutions  of  (2.1)  are  globally  uniformly  bounded  and 
satisfy 

lim  W{x{t))  0  (2.11) 

In  addition,  if  W(x)  is  positive  definite,  then  the  origin  is  globally,  uniformly  asymptotically  stable. 

Proof:  See  [40] ,  Theorem  A.8.  ■ 

A  converse  form  of  Theorem  2.1.1  exists  which  allows  us  to  prove  instability  of  an  equilibrium. 


Theorem  2.1.3  (Converse  Lyapunov)  Let  x  =  0  be  an  equilibrium  of  (2.1)  and  let  U{0,r)  G  TZ'^ 
be  an  open  ball  of  radius  r  G  7?.+  centered  at  the  origin  as  defined  above.  Also  letV  :U  TZ^  be  a 
continuously  differentiable  positive  definite  function  V{x)  on  U,  and  W{x)  continuous  and  positive 
definite  on  U  such  that 

dv 


V  =  ^^a{x,t)  >  W{x)  >  0  V  t  X  in  some  neighborhood  N of  the  origin 

ax 


(2.12) 
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Then  the  origin  is  an  unstable  equilibrium  of  (2A). 


Proof:  See  [68],  Theorem  8.3.  ■ 

If  we  want  to  consider  only  time-invariant  nonlinear  systems 


X  —  a{x)  (2.13) 

then  LaSalle’s  Invariance  Theorem  and  its  associated  asymptotic  stability  theorem  will  also  prove 
useful.  A  prerequisite  is  the  definition  of  an  invariant  set.  A  set  M  C  'RT'  is  said  to  be  invariant 
with  respect  to  (2.13)  if,  for  some  t^  >  0, 

x{to)  =  xo  ^  M  ^  x{t]  xq,  to)  e  M  \/  t  e  7^+  (2.14) 

while  M  is  said  to  be  positively  invariant  if 

x{to)  =  xq  e  M  ^  x{t;  Xo^to)  e  M  W  t  >  to  (2.15) 

We  now  present  the  theorems. 

Theorem  2.1,4  (Lasalle  Invariance)  Let  Q  be  a  positively  invariant  set  of  (2 A3)  and  let  V  : 
ft  TZ^  be  a  continuously  differentiable  function  V{x)  such  that  V{x)  <  0  M  x  E  ft.  Also^  let 
E  =  {x  E  ft  \V{x)  ~  0};  and  let  M  be  the  largest  invariant  set  contained  in  E,  Then  every  bounded 
solution  x(i)  starting  in  (1  converges  to  M  as  t  oo. 

Proof:  See  [44].  ■ 

Theorem  2.1.5  (Time-invariant  GAS)  Let  x  =  0  be  the  only  equilibrium  of  (2A3)  and  let  V  : 
RT'  —>  be  a  continuously  differentiable,  positive  definite,  radially  unbounded  function  V (x)  such 
that  V{x)  <  0  \/  x  E  Also,  let  E  ~  {x  E  RT'  \  V{x)  =  0},  and  suppose  no  solution  other  than 
x{t)  —  0  can  stay  forever  in  E.  Then  the  origin  is  globally  asymptotically  stable  (GAS). 
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Proof:  See  [44]. 


In  later  sections  of  this  dissertation,  we  will  want  to  consider  stability  of  systems  with  inputs. 
Thus,  we  need  a  final  stability  concept,  introduced  by  Sontag  [65].  A  system 

x:=f{x,u)  (2.16) 

is  said  to  be  input-to-state  stable  (ISS)  if  for  any  x(0)  and  for  any  input  u  continuous  and  bounded 
on  [0,  oo)  the  solution  to  (2.16)  exists  V  i  >  0  and  satisfies 

||a:(i)||  <  /3(||x(0)||,  t)  +  7  f  sup  ||u(r)||)  ,  V  i  >  0  (2.17) 

\0<r<<  / 

where  ^{s^t)  and  7(5)  are  strictly  increasing  functions  of  s  G  TZ-\.  with  /3(0,  £)  =  0,7(0)  =  0,  while 
/J  is  a  decreasing  function  of  t  with  /3— >0ast^oo,  VsG  This  definition  is  appropriate 
for  nonlinear  systems  in  that  it  includes  effects  of  the  control  and  of  the  initial  condition.  The  ISS 
nature  of  a  system  (2.16)  can  be  established  by  at  least  two  methods  other  than  simply  using  the 
definition:  via  a  closed  loop  Lyapunov  type  argument  and  via  a  dissipation  inequality  argument. 
These  are  presented  in  the  following  theorem,  but  first  we  present  some  useful  definitions. 

A  continuous  function  p  :  [0,a)  is  said  to  be  of  class  JC  if  it  is  positive  definite  and  strictly 

increasing.  It  is  said  to  be  of  class  ICqo  if,  in  addition,  a  =  00  and  it  is  radially  unbounded. 

Theorem  2.1,6  Suppose  that  for  (2A6)  3  a  continuously  differentiable  class  /Coo  function  V  : 
TiA'  X  71^  — >  7^^.^  class  JCoo  functions  k,  pi  and  p2,  and  a  class  JC  function  ijj  such  that 

i  'i  X  £  and  u  E 

ll^ll  >  «(ll^ll)  ^  (2-18) 

or, 

ii. 

^f{x,u)<-p^(\\x\\)+pM)  (2.19) 

Then  (2.16)  is  ISS,  and  if  item  i  above  holds  then  7  in  (2.17)  is  equal  to  k,  and  we  call  V  in  (2.18) 

an  ISS-Lyapunov  function. 
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2.1.2  Stability  in  the  First  Approximation  and  Center  Manifold  Theory 

In  the  latter  part  of  Section  2.1.1,  we  presented  two  theorems  based  on  Lyapunov  arguments  that 
are  useful  for  determining  the  stability  of  equilibria  of  nonlinear  time  invariant  systems  described 
by  (2.13).  Another  well  known  test  for  determining  local  stability  of  equilibria  of  (2.13)  is  the  so- 
called  Principle  of  Stability  in  the  First  Approximation.  If  we  assume  that  a  in  (2.13)  is  at  least 
twice  continuously  differentiable  and  that  the  equilibrium  of  interest  is  again  the  origin,  then  the 
following  statements  can  be  made  about  the  local  asymptotic  stability  of  (2.13)  based  on  the  linear 
approximation  of  a  at  a;  =  0  [68].  Let 


Jo  = 

denote  the  Jacobian  matrix  of  a  at  a;  =  0.  Then  if 


da 

dx 


J  31=0 


(2.20) 


•  all  the  eigenvalues  of  Jo  have  negative  real  parts,  the  origin  is  a  locally  asymptotically  stable 
equilibrium  of  (2.13), 

•  at  least  one  eigenvalue  of  Jo  has  a  positive  real  part,  the  origin  is  an  unstable  equilibrium  of 
(2.13). 

The  Principle  of  Stability  in  the  First  Approximation  obviously  does  not  cover  all  cases  of  in¬ 
terest.  In  particular,  it  provides  no  information  when  all  of  the  real  parts  of  the  eigenvalues  of  Jo 
are  nonpositive,  and  at  least  one  eigenvalue  has  a  zero  real  part.  When  this  is  the  case.  Center 
Manifold  Theory  may  often  be  used  to  draw  conclusions  regarding  the  local  stability  properties  of 
an  equilibrium  point  for  a  time-invariant  system.  We  point  out  that  entire  texts  have  been  written 
on  this  topic,  and  in  order  to  keep  the  level  of  detail  manageable,  we  will  present  just  enough  of  the 
theory  to  meet  our  analysis  purposes.  This  material  is  extracted  from  [33],  Appendices  A  and  B,  to 
which  the  reader  is  referred  for  a  more  comprehensive  treatment  of  the  subject.  We  will  again  need 
a  few  definitions  before  proceeding. 
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Recall  that  an  open  set  O  in  TV^  is  defined  as  a  collection  of  points  x  E  TiP  such  that  for  every 
X  E  O  a.n  open  ball,  B{x^r),  exists  which  is  wholly  contained  in  O  [51].  Unless  stated  otherwise, 
when  we  use  the  term  smooth  manifold,  we  are  simply  referring  to  some  special  open  subset  of 
7?.^.  In  fact,  any  open  subset  of  is  a  smooth  manifold  of  dimension  n  [33].  With  this  in  mind, 
our  earlier  definition  of  an  invariant  set  readily  extends  in  the  expected  way  to  include  the  term 
invariant  manifold.  However,  to  define  a  center  manifold,  we  need  the  concept  of  a  locally  invariant 
manifold.  A  manifold  N  is  said  to  be  locally  invariant  with  respect  to  (2.13)  if  for  each  Xq  €  iV, 
3  >  0  such  that  x{t;xQ,0)  e  N  \/  t  e  (0,ii)  (note  that  we  have  arbitrarily  set  =  0,  which  we 

can  do  without  loss  of  generality  since  (2.13)  is  time-invariant).  Another  required  definition  is  that 
of  the  tangent  space  of  a  smooth  manifold  at  a  point.  This  concept  requires  some  development, 
presented  below. 

Let  Nhea.  smooth  manifold  of  dimension  n.  The  vector  space  of  all  functions  defined  on  TZ^  that 
are  r  times  continuously  differentiable  we  shall  denote  by  C^.  We  shall  say  a  function  is  smooth  if 
it  is  infinitely  times  continuously  differentiable.  Thus,  the  vector  space  of  all  functions  defined  on 
TZ"^  that  are  infinitely  times  continuously  differentiable  we  shall  denote  by  .  Now,  let  p  be  any 
point  in  iV.  A  real- valued  function  A  is  said  to  be  smooth  in  a  neighborhood  of  p  if  the  domain 
of  A  includes  an  open  subset  U  oi  N  containing  p  and  the  restriction  of  A  to  U  is  a  smooth  function 
as  defined  above.  We  denote  the  set  of  all  functions  smooth  in  a  neighborhood  of  p  by  C^{p)^ 
noting  that  C°°{p)  is  a  vector  (linear)  space  over  the  field  TZ^  and  in  fact  C^(p)  is  a  commutative 
linear  algebra  since  multiplication  of  vectors  is  well-defined  and  order-independent.  We  now  give 
the  definition  of  a  tangent  vector  at  a  point  in  a  smooth  manifold. 

A  tangent  vector  -y  at  p  is  a  map  v  :  C^{p)  1Z  satisfying  the  following  two  properties: 

i.  (linearity):  v{aX  H-  by)  —  av{X)  -j-  bv{^)  V  A,7  E  C°°{p)  and  a^b  E  TZ 

ii.  (Leibniz  Rule):  y(A7)  =  7(p)y(A)  -h  X{p)v{^)  V  A,  7  E  C°°(p) 

We  define  the  tangent  space  to  N  at  p,  written  TpAT,  as  the  set  of  all  tangent  vectors  at  p.  It 
is  simple  to  show  that  the  set  TpN  forms  a  vector  space  over  72<,  with  the  standard  rules  of  vector 
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addition  and  scalar  multiplication.  The  tangent  space  to  a  smooth  manifold  N  at  the  point  p  may 
be  thought  of  geometrically  as  a  “tangent  hyperplane”  to  N  at  p.  We  may  construct  a  basis  for  the 
tangent  space  according  to  the  following  lemma: 

Lemma  2,1.1  Let  N  be  a  smooth  manifold  of  dimension  n.  Let  p  be  any  point  of  N.  The  tangent 
space  TpN  to  N  at  p  is  an  n-dimensional  vector  space  over  the  field  Tl.  If  . . .  ,(^n)  ^  set  of 

coordinates  for  N  around  p,  then  the  tangent  vectors  ,  (af“)p  ^  basis  of  TpN,  called 

the  natural  basis  induced  by  the  coordinates  (0i, . . . ,  ^n)* 

Lemma  2.1.1  provides  a  means  of  interpreting  r^(A),  where  t;  is  a  tangent  vector  at  the  point  p  and 
A  is  a  smooth  function  in  the  neighborhood  of  p.  Let  N  ~  and  let  p^  N.  K  we  choose  the  <j>i  in 
Lemma  2.1.1  to  be  the  standard  coordinate  set  in  i.e.,  f)i  =  [xi  0. ..  0]^,  •  ^•,(t>n  ~  [0 ...  0  Xn]'^, 
then  from  the  definition  of  a  basis  we  can  write  t;(A)  —  Yh=i 

t;(A)  can  be  seen  to  be  the  value  of  the  derivative  of  A  along  the  direction  of  the  vector  [vi . .  .Vn]^ 
at  the  point  p. 

Before  defining  a  center  manifold,  we  need  a  few  facts  from  linear  algebra.  Suppose  the  Jacobian 
matrix  Jo  of  (2.13)  has  n^  (c  for  center  manifold)  eigenvalues  with  zero  real  part,  n^  {s  for  stable 
manifold)  eigenvalues  with  negative  real  part,  and  n'^  {u  for  unstable  manifold)  eigenvalues  with 
positive  real  part.  It  is  well  known  from  linear  algebra,  e.g,  see  [28],  that  the  domain  of  the  linear 
mapping  F  can  be  decomposed  into  the  direct  sum  of  three  invariant  subspaces,  noted  ,  and 

E^  (with  respective  dimensions  n^,  n^,  and  n^),  with  the  property  that  Jo  |je;^  (Jo  with  its  domain 
restricted  to  E^)  has  all  eigenvalues  with  zero  real  part,  Jo  \e^  has  all  eigenvalues  with  negative 
real  part,  and  Jo  \e^  all  eigenvalues  with  positive  real  part.  Furthermore,  it  is  true  that  the 
direct  sum  of  these  three  subspaces  equals  the  tangent  space  to  TV  at  a;  —  0,  TqN,  We  now  have  the 
machinery  in  place  to  define  a  center  manifold. 

Let  a;  ==  0  be  an  equilibrium  of  (2.13).  A  manifold  5,  passing  through  a;  =  0,  is  said  to  be  a 
center  manifold  for  (2.13)  at  a;  =  0,  if  it  is  locally  invariant  and  the  tangent  space  to  S  at  zero  is 
exactly  E^, 
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In  the  remainder  of  this  section,  we  will  be  interested  in  using  Center  Manifold  Theory  to 
determine  if  an  equilibrium  is  stable  or  not.  By  the  Principle  of  Stability  in  the  First  ApproMmation, 
we  can  declare  an  equilibrium  unstable  if  Jq  has  any  eigenvalues  with  positive  real  part.  Thus,  we 
will  assume  that  Jq  has  eigenvalues  with  only  zero  or  negative  real  parts.  Under  this  assumption, 
we  can  write  (2.13)  in  the  form 


y  =  Ay  +g{y,z)  (2.21) 

z  =  Bz  h{y^  z)  (2.22) 


where  the  eigenvalues  of  the  matrix  A  all  have  negative  real  part  and  the  eigenvalues  of  the  matrix 
B  all  have  real  part  equal  to  zero,  by  performing  the  following  actions.  Write  a{x)  =  JqX  -h  d(a;), 
and  then  reduce  Jq  to  a  block  diagonal  form 


r-VoT- 


(2.23) 


by  letting  the  columns  of  the  matrix  T  be  the  right  eigenvectors  of  Jq  and  defining  the  linear 
coordinate  transformation 


/  \ 


y 


^T-'^x 


(2.24) 


The  existence  of  center  manifolds  for  (2.21),  (2.22)  is  guaranteed  by  the  following  theorem  (recall  in 
(2.13)  that  a  G  >  2). 


Theorem  2.1.7  There  exist  a  neighborhood  U  C  of  z  —  0  and  a  ^  mapping  tt  :  U 
such  that 

S  =  {{y,  z)  G  X  t/  I  y  =  7r(z)}  (2.25) 

is  a  center  manifold  for  (2.21)^  (2.22). 


Proof:  See  [10].  ■ 

Although  existence  of  a  center  manifold  for  (2.21),  (2.22)  is  guaranteed  by  Theorem  2.1.7,  unique¬ 
ness  is  not,  and  in  general,  many  center  manifolds  for  a  system  are  possible.  Nevertheless,  the  above 
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definition  and  Theorem  2.1.7  are  sufficient  to  characterize  the  critical  behavior  of  any  center  mani¬ 
fold  for  (2.21),  (2.22).  By  definition,  a  center  manifold  passes  through  (0,0)  and  is  tangent  to  at 
a;  —  0.  The  tangency  condition  is  equivalent  to  the  mapping  tt  having  zero  slope  in  the  directions  of 
the  ^  coordinates.  Thus,  tt  must  satisfy 

dir 

7r(0)  =  0 ,  ^(0)  =  0  (2.26) 

The  locally  invariant  nature  of  S  places  another  constraint  on  the  mapping  tt.  Since  y{t)  =  '^{z{t)) 
everywhere  on  5,  we  can  differentiate  this  expression  and  use  (2.21),  (2.22)  to  obtain 

I  =  Aniz(t))+9{n{z{t)),z{t))  =  =  ^(Bz(t)  +  h(vr(zm  z(t)) 

Thus,  TT  must  satisfy  the  partial  differential  equation 

/)'7r 

—  (Bz  +  h(7r(z),z))  =  A-!r(z)  +  g(7r(z),  z)  (2.27) 

Characterization  of  the  system’s  behavior  on  the  center  manifold  is  critical  because  it  can  be 
shown  that  all  system  trajectories  which  begin  in  a  neighborhood  of  the  center  manifold  will  approach 
it  with  exponential  convergence  [33].  Thus,  system  behavior  on  the  center  manifold  determines  local 
stability  of  the  equilibrium  point  in  question.  This  behavior  can  be  determined  by  solving  (2.22) 
with  y  set  equal  to  7r(^),  that  is,  solving 

z  ~  Bz  +  h{'K{z),  z)  (2.28) 

This  is  formalized  in  the  following  theorem. 

Theorem  2, 1.8  (Reduction  Principle)  Suppose  z  ~  0  is  a  stable  (respectively  asymptotically 
stable,  unstable)  equilibrium  of  (2.28),  Then  {y,z)  ~  (0,0)  is  a  stable  (respectively  asymptotically 
stable,  unstable)  equilibrium  of  (2.21),  (2.22). 

Proof:  See  [10].  ■ 
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This  theorem  is  called  the  Reduction  Principle  because  it  reduces  the  dimension  of  the  system 
that  must  be  studied  to  determine  stability  of  (2.21),  (2.22)  from  n  to  which  can  be  a  major 
simplification.  However,  (2.28)  must  still  be  solved  or  its  stability  determined  in  some  way.  The 
Lyapunov  theorems  of  Section  2.1.1  provide  one  tool  for  doing  so.  Alternatively,  we  introduce  several 
other  helpful  theorems  and  lemmas  in  the  next  section. 


2 A, 3  Useful  Theorems  and  Lemmas 

Application  of  Lyapunov  and  center  manifold  theory  to  systems  of  various  assumed  general  structures 
allow  characterization  of  stability  for  these  systems.  We  present  several  of  these  well  known  results 
here  [33],  to  aid  in  later  stability  analyses. 


Lemma  2.1.2  Let  y  =  7Ck{z)  he  a  polynomial  of  degree  Aj,  1  <  <  r,  satisfying 


TTfeCO)  =  0  , 


dTTk 

dz 


(0)  =  0 


and  suppose 


d'Kk 

dz 


{Bz  +  h(7rk{z),  z))  -  ATVkiz)  -  g{7rk{z),  z)  =  Rk{z) 


where  Rk  is  some  (possibly  unknown)  function  vanishing  at  0  together  with  all  partial  derivatives  of 


order  less  than  or  equal  to  k.  Then,  any  solution  7r{z)  of  (2,27)  is  such  that  the  difference 


Dk{z)  =  t{z)  - 


vanishes  at  0  together  with  all  partial  derivatives  of  order  less  than  or  equal  to  k. 


Lemma  2.1.2  allows  us  to  use  the  next  lemma  in  determining  stability  of  center  manifold  equa¬ 
tions. 


Lemma  2.1.3  Consider  the  one- dimensional  system 

X  =  kx"^  -f-  Qm{^) 


2-12 


with  m  >2,  0  k  eTl  and  Qm{^)  «  function  vanishing  at  zero  together  with  all  partial  derivatives 
of  order  less  than  or  equal  to  m.  The  point  of  equilibrium  x  =  0  is  asymptotically  stable  if  m  is  odd 
and  k  <0.  The  equilibrium  is  unstable  if  m  is  odd  and  k  >  0,  or  if  m  is  even. 

We  next  present  several  stability  analysis  tools  for  systems  of  the  form  of  (2.21),  (2.22),  with 
various  assumptions  on  g  and  h. 

Theorem  2.1,9  Consider  a  system 

y=Ay+g{y,z) 

z  =  h{y,z)  (2.29) 

and  suppose  that  p(0,  z)  —  0  z  near  0  and 

If(0.0)  =  0 

If  z  —  /i(0,  z)  has  an  asymptotically  stable  equilibrium  at  z  =  0  and  the  eigenvalues  of  A  all  have 
negative  real  part,  then  the  system  (2.29)  has  an  asymptotically  stable  equilibrium  at  {y,z)  —  (0,0). 

Proof:  See  [33],  pg.  512.  ■ 

Lemma  2.1.4  Consider  a  system 

y  =  giy) 

z  =  h{y,  z)  (2.30) 

and  suppose  that  y  =  g{y)  has  an  asymptotically  stable  equilibrium  at  y  ~  0.  If  z  —  h{Q,  z)  has 
an  asymptotically  stable  equilibrium  at  z  =  0,  then  the  system  (2.30)  has  an  asymptotically  stable 
equilibrium  at  {y,  z)  =  (0,0). 

Lemma  2.1.5  Consider  a  system 

y  =  9{y) 

z  =  h{y,z)  (2.31) 
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and  suppose  that  {y,z)  -  (0,0)  is  an  equilibrium  of  (2.31),  y  =  g{y)  has  a  stable  equilibrium  at 
y  ==  0,  and  z  —  /i(0,  z)  has  an  asymptotically  stable  equilibrium  at  z  —  0.  Then  the  equilibrium 
{y,z)  =  (0,0)  of  (2.31)  is  stable. 

We  conclude  this  section  by  noting  that,  in  all  situations  where  an  asymptotically  stable  equilib¬ 
rium  is  required,  the  equilibrium  does  not  necessarily  have  to  be  stable  in  the  first  approximation, 
i.e.,  have  a  Jacobian  with  all  eigenvalues  in  the  open  complex  left-half  plane.  Thus,  center  manifold 
or  Lyapunov  theory  may  be  required  to  determine  the  asymptotic  stability  of  an  equilibrium,  prior 
to  invoking  the  above  theorems  and  lemmas. 

2.2  Feedback  Linearization 

This  section  provides  a  summary  of  the  theory  needed  to  conduct  nonlinear  control  design  via  the 
technique  of  feedback  linearization,  also  known  as  dynamic  inversion.  In  this  section  we  will  consider 
input-affine  multivariable  nonlinear  control  systems  in  state  space  form,  i.e., 

m 

X  =  a{x)  +  bi{x)ui 

i=i 

y  =  c{x)  (2.32) 

where  x  e  U  C  'RT',  u  E  RF,  and  y  E  The  mappings  a, are  smooth,  valued 
mappings  defined  on  the  open  set  [/,  and  may  be  represented  by  n-dimensional  vectors  of  real¬ 
valued  functions  of  the  real  variables  xi,...,Xn.  A  similar  statement  holds  for  c  except  p  replaces 
n.  Thus,  we  can  write 


ai{xi,.. 

•  )  ^n) 

b\i{X\ ,  .  . 

\ 

•  j  ^n) 

^  Clixi,.. 

»  5  ^n) 

a{x)  — 

^  an{x^i  5  *  • 

•  }  ^n)  j 

,  bi{x^  — 

^  bniip^l  ?  *  • 

•  5  ^n)  J 

,  c{x)  = 

^  Cp{Xi,  . . 

•  5  ^n)  j 

(2.33) 

We  begin  this  section  with  definitions  of  needed  terms.  We  then  give  the  basic  theory  for  single¬ 
input,  single-output  (SISO)  systems  in  Section  2.2.2,  and  extend  the  results  to  the  multi-input, 
multi-output  (MIMO)  case  in  Section  2.2.3.  The  bulk  of  the  material  is  extracted  from  [33]. 
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2.2.1  Terms  and  Definitions  from  Differential  Geometry 


Since  a  and  hi  (cj)  in  (2.32)  above  map  each  point  x  of  {7  to  a  vector  in  they  are  often 

called  vector  fields  defined  on  U.  Similarly,  a  covector  field  defined  on  U  is  identified  as  a  1  x  n 
(i.e.  row)  vector  of  functions  of  x  (called  a  covector  due  to  its  association  with  the  dual  space  of 
TZ^).  A  special  covector  of  importance  is  the  differential,  or  gradient,  of  a  real- valued  function  A 
(also  assumed  to  be  defined  on  C/),  which  is  defined  in  the  standard  way 


d\{x)  =  g  = 


dX 

dxi  ‘  ‘  *  dXn 


(2.34) 


We  now  define  three  types  of  dififerential  operations  involving  smooth  functions,  vector  fields,  and 
covector  fields  used  frequently  in  the  analysis  of  nonlinear  control  systems  and  central  to  feedback 
linearization  theory.  Let  A  and  a  be  defined  as  above.  Then  the  Lie  derivative  of  A  along  a  (or 
with  respect  to  a),  written  LaX,  is  the  smooth  real-valued  function  defined  by 


LaX{x)  =  {dX^(x),  a{x))  =  ^a{x)  =  ^^x)  (2.35) 

for  each  x  where  (♦,  •)  denotes  the  standard  inner  or  dot  product  on  7?.”,  and  superscript  “T” 
denotes  the  transpose  operator.  Note  that  this  convention  could  have  been  used  in  all  the  theorems 
of  Section  2.1.1,  by  replacing  V  with  LaV.  Also  note  that  repeated  use  of  this  operation  is  possible, 
so  that  for  example 

LgLaX{x)  =  ^-^9{x) 

If  A  is  being  differentiated  k  times  along  a,  the  notation  L^A  is  used,  with  L^A  =  A. 

The  second  type  of  operation  involves  two  vector  fields  a  and  both  defined  on  an  open  subset 
U  of  'R^' .  From  these  we  construct  a  new  smooth  vector  field  called  the  Lie  bracket  or  Lie  product 
of  a  and  denoted  [a,  g]  (or  adag)^  and  defined  at  each  x  6  C/  as 

In  (2-36)  above,  dajdx  and  dg/dx  represent  the  Jacobian  matrices  of  a  and  respectively.  Repeated 
bracketing  of  g  with  a  is  possible  and  can  be  defined  recursively  by 


adaQix)  =  [a,  adl  ^g]{x) 
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for  any  k  >  1,  setting  CLd°g{x)  =  g(x). 

If  ai,gi,  and  ;p  are  vector  fields  and  Vi  real  numbers,  it  is  easily  proven  that  the  Lie  bracket 
operation  possesses  the  following  three  properties 

i.  (Bilinearity) 

[rioi  +r2a2,ffi]  -  n[ai,5i]  +  (2.37) 

[oi,n5i  +’•252]  =  n[ai,</i]  +»’2[oi,<72]  (2.38) 

ii.  (Skew- Commutativity) 

[a,g]  =  -[g,a]  (2.39) 

iii.  (Jacobi  Identity) 

[a,  [fif,  P]]  +  [ff,  [P,  a]]  +  b.  [a,  fl]]  =  0  (2.40) 

The  third  type  of  operation  involves  a  covector  field  w  and  a  vector  field  v  defined  on  an  open 
subset  U  of  It  produces  a  new  covector  field,  called  the  derivative  of  w  along  a,  written 
and  defined  for  each  x  6  as 

La2«(x)  =  a^(a;)  (^)  +w{x)^  (2.41) 

Another  important  concept  is  that  of  a  distribution.  Suppose  we  are  given  I  smooth  vector 
fields  tti, . . ,  ,a^,  with  p  elements  each,  all  defined  on  an  open  subset  U  of  If  we  evaluate  each 
ai  at  some  a;  G  17,  we  obtain  I  vectors,  the  span  of  which  forms  a  subspace  of  TZ^.  This  smooth 
assignment  of  a  point  in  [7  to  a  subspace  of  TZ^  defines  the  rule  for  evaluation  of  a  distribution,  A, 
i.e., 

A(a;)  span{ai(x), . . .  ,a;(x)}  V  X  €  ?7  (2.42) 

and  we  use  the  notation 

A  span{ai,...,az} 

to  define  A  itself.  Addition,  intersection,  and  containment  operations  and  relationships  are  defined 
pointwise  for  distributions  according  to  normal  subspace  rules.  Also,  we  say  a  belongs  to  A,  and 
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write  a  €  A,  if  a{x)  e  A(a;)  \/  x  eU.  The  dimension  of  a  distribution  at  a  point  x  eU^  dim(A(a;)), 
is  the  dimension  of  the  subspace  A(a^).  A  distribution  A,  defined  on  an  open  set  I/,  is  said  to  be 
nonsingular  if  there  exists  an  integer  d  such  that 

dim(A(a?))  =  dV  x  eU 

A  point  x^  of  U  is  said  to  be  a  regular  point  of  a  distribution  A,  if  there  exists  a  neighborhood 
of  x^  with  the  property  that  A  is  nonsingular  on  [7®,  A  point  of  U  which  is  not  a  regular  point 
is  called  a  point  of  singularity. 

The  following  lemma  illustrates  the  utility  of  a  regular  point  of  a  distribution. 

Lemma  2.2.1  Let  A  be  a  smooth  distribution  andx^  a  regular  point  of  A,  Suppose  dim(A{x^))  ■=  d. 
Then,  3  a  neighborhood  ofx^  and  a  set  ai, . . .  ,ad  of  smooth  vector  fields  defined  on  such  that 

L  the  vectors  ai{x)^ . . . ,  ad[x)  are  linearly  independent  at  each  x  £ 

a.  A(x)  =:  span{ai{x)^ . , ,  ^ad{x)}  \f  x  e 

in,  every  smooth  vector  field  r  E  A  can  be  expressed  on  as 

d 

■^(^)  =  53ci(a:)ai(3;) 

2=1 

where  ci(x), ,, .  yCd{x)  are  smooth  real-valued  functions  of  x,  defined  on  U^, 

A  distribution  A  is  said  to  be  involutive  if  the  Lie  bracket  [ri,  T2]  of  any  pair  of  vector  fields  ti 
and  T2  belonging  to  A  is  a  vector  field  which  belongs  to  A,  i.e.,  if 

Ti  E  A,T2  E  A  ^  [ri,  r2]  E  A 

Thus,  if  A  is  nonsingular,  A  is  involutive  iff 

[ai^aj]  E  A  \f  i  >  l,j  <  d  (2.43) 

Because  of  (2.43),  checking  whether  or  not  a  nonsingular  distribution  is  involutive  amounts  to 
checking  if 

rank(ai(a;) . .  .ad{x))  =  miik{ai{x) . . .  aci{x)  [a*,  afi)  W  x,\/  i  >lj<d 
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We  note  that  a  simple  consequence  of  the  above  definitions  is  that  any  1-dimensional  distribution  is 
involutive. 

All  of  the  above  terms  and  definitions  can  similarly  be  posed  in  terms  of  covectors,  thus  resulting 
in  codistributions.  A  useful  example  is  the  annihilator  of  the  distribution  A(x),  defined  by 

A^{x)  -  {w^  e  {TTf  :  {w\v)  =  0  V  r;  e  A{x)] 

and  for  which  the  following  lemma  applies. 

Lemma  2.2.2  Let  A  be  a  smooth  distribution  and  a  regular  point  of  A.  Then  is  a  regular 
point  of  A-^  and  3  a  neighborhood  of  x^  such  that  the  restriction  of  A^  to  is  a  smooth 

codistribution, 

A  perhaps  somewhat  expected  fact  is  that  dim(A(a;))  +  dim(A-^(a;))  =  n. 

As  we  have  now  built  most  of  the  necessary  vocabulary,  in  the  next  section  we  proceed  to 
introduce  the  basic  concepts  and  theorems  involved  in  feedback  linearization. 

2,2,2  Theory  for  Single-Input  Single- Output  Systems 

In  this  section  we  present  the  basic  theory  of  feedback  linearization  for  SISO  nonlinear  dynamic 
systems.  As  mentioned  in  Section  1.1,  the  basic  idea  is  to  introduce  a  nonlinear  change  of  coordinates 
and  static  state  feedback  which  renders  the  closed  loop  system  linear  and  stabilizable,  enabling 
completion  of  the  control  design  by  application  of  any  suitable  linear  design  technique  to  the  resulting 
system.  Thus,  we  introduce  some  formal  concepts  about  nonlinear  coordinate  changes  in  Section 
2.2. 2.1.  Following  that,  we  address  the  two  cases  in  which  at  least  some  degree  of  linearization 
is  possible:  exact  (total)  linearization  of  the  input  to  state  response,  and  partial  (input-output) 
linearization  resulting  in  the  so-called  zero  dynamics. 

2,2, 2.1  Local  Coordinate  Transformations 
A  global  nonlinear  change  of  coordinates  can  be  described  in  the  form 

=  ^{x) 
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where  ^(x)  represents  an  valued  function  of  n  variables,  i.e., 


\ 

,  \ 

(f)l{x) 

•  1  ^n) 

^  4>n{x)  j 

^  071  (^1  7  •  • 

•  5  j 

^x)  = 


with  the  following  properties 

i.  $(a;)  is  invertible,  i.e.  there  exists  a  function  such  that 

$“^($(2;))  =  x\/  X  £  TZ^ 


(2.44) 


ii.  $(a:)  and  are  both  smooth  mappings  (have  continuous  partial  derivatives  of  any  order) 

Such  a  global  transformation  is  called  a  global  diffeomorphism  on  If  (2.44)  above  only  holds 
on  some  neighborhood  of  a  point  of  interest,  the  transformation  $  is  called  a  local  diffeomorphism. 
The  following  theorem  provides  a  means  of  checking  whether  or  not  a  given  mapping  is  a  local 
diffeomorphism. 

Theorem  2.2*1  (Inverse  Function)  Suppose  ^(rc)  is  a  smooth  function  defined  on  some  subset 
U  ofTZ^.  Suppose  the  Jacobian  matrix  of  ^  is  nonsingular  at  a  point  x^  £U,  Then,  on  a  suitable 
open  subset  ofU  containing  x^,  $(a;)  defines  a  local  diffeomorphism. 

Proof:  See  [1],  Theorem  13.6.  ■ 

We  now  focus  our  attention  on  obtaining  conditions  for  existence  of  a  special  type  of  diffeomor¬ 
phism  to  achieve  our  goal  of  feedback  linearization.  Prior  to  proceeding,  we  need  one  more  definition. 
Recall  Equation  (2.32),  where  now  m  —  p  —  1,  corresponding  to  our  assumption  of  a  SISO  nonlinear 
dynamic  system.  Such  a  system  is  said  to  have  relative  degree  r  at  a  point  if 

i.  LtL^c{x)  —  0  V  X  in  a  neighborhood  of  x^  and  \/  k  <  r  ~  1 

ii.  Li,Ll-^c{x^)  ^  0 
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We  note  that  there  may  exist  points  where  a  relative  degree  cannot  be  defined.  However,  the  set  of 
points  where  it  is  defined  is  an  open  and  dense  subset  of  the  set  U  for  which  (2.32)  is  defined  [33]. 

Intuitively  speaking,  the  relative  degree  of  a  SISO  system  is  the  number  of  times  the  output 
y  must  be  differentiated  for  the  input  to  appear  explicitly  on  the  right  hand  side  of  the  resulting 
expression.  This  can  easily  be  seen  from  simple  construction,  i.e.,  given  =  x{t^)  we  have 

y{f)  =  c(a;(«“))  =  ^a:") 

=  If  = 

=  Lac{x{t))  +  Lbc{x{t))u{t)  (2.45) 


SO  that 

—  L^c{x{t))  V  k  <  r,\/  t  near  f 

=  Llc{x^)+LbU-^c{x°)u{t°)  (2.46) 

We  also  observe  that  if 

LbL^c{x)  =  0V^>0,Vrcina  neighborhood  of  x^ 

then  the  output  is  not  affected  by  the  system  input  for  all  t  near  and  no  relative  degree  can  be 
defined  at  any  point  around  x^.  The  concept  of  relative  degree  plays  a  central  role  in  determining 
when  we  may  find  a  local  diffeomorphism  such  that  the  output  y  —  c(x)  is  a  state  variable  in  the 
new  set  of  coordinates^  as  indicated  by  the  following  lemma  [33] . 

Lemma  2.2.3  Suppose  the  system  (2. 32)  with  m  —  p  ~  1  has  relative  degree  r  at  x^ .  Then  r  <  n. 
Set 


(j)i{x)  =  c{x) 

(/)2(x)  =  LaC{x) 

<t>r{x)  =  U--^c{x)  (2.47) 
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//r  <  n,  3  n  —  r  more  functions  ,  0n(x)  suck  that  the  mapping 


( 


(j)l{x) 


\ 


#(2;)  = 


y  <i>n{x)  j 


has  a  nonsingular  Jacobian  matrix  at  .  The  value  of  these  additional  functions  at  x^  can  be  fixed 
arbitrarily.  Moreover,  it  is  always  possible  to  choose  (pr+i{x),  •  •  - ,  (fnix)  such  that 


Lb(j>i{x)  =  0  \/  r  +  1  <  i  <  n,W  x  near  a;° 


(2.48) 


From  Theorem  2.2.1,  we  see  that  $  in  Lemma  2.2.3  qualifies  as  a  local  diffeomorphism.  In  concluding 
this  section,  we  can  say  something  more  about  the  form  of  the  system  equations  in  the  new  set  of 
coordinates  obtained  under  the  mapping  #.  If  (2.48)  is  satisfied,  then  the  resulting  state-space 
system  description  in  the  new  set  of  coordinates  is  of  the  form 


Zi  = 


Z2  =  Z3 


Zf  —  1  —  Z'p 


Zr  =  P{z)  +  a{z)u 


^ri-1  —  Qr+l  (^) 


i^n  =  qn{z)  (2.49) 

and  we  see  that  the  first  r  -  1  state  equations  are  a  chain  of  integrators,  the  rth  state  equation  has 
the  control  entering  directly,  and  the  last  n-r  state  equations  show  no  direct  effect  of  the  control. 
A  set  of  state  equations  in  the  form  of  (2.49)  above  is  said  to  be  in  normal  form.  Lemma  2.2.3 
states  that  it  is  always  possible  to  satisfy  (2.48),  but  in  fact  this  may  not  always  be  easy.  As  per 
Theorem  2.2.1,  if  all  we  desire  is  a  local  diffeomorphism,  we  may  in  fact  choose  . . . , 
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arbitrarily  so  long  as  the  Jacobian  of  ^  at  is  nonsingular.  If  we  do  so,  however,  the  resulting 
state  equations  will  in  general  have  the  form 

Zi  ^  Z2 

h  ^  ^3 


Zr~l  =  Zr 

Zr  =  /3{z)  -h  a{z)u 


Zr^l  =  qr^l{z) pr+l{z)u 


Zn  ^  Quiz)  +  Pn{z)u  (2.50) 

where  now  u  enters  the  last  n—r  state  equations  explicitly.  The  benefits  of  choosing  0r+i  •  •  •  ?  0n(^) 
according  to  (2.48)  will  be  seen  in  the  next  section,  as  we  proceed  toward  obtaining  a  linearized 
input-output  relationship. 

2. 2, 2. 2  Exact  Linearization  Via  Feedback 

If  we  examine  (2.49)  a  little  more  closely,  we  see  that  for  a  SISO  system  with  relative  degree  r 

in  normal  form,  the  subsystem  associated  with  the  first  r  —  1  transformed  coordinates  is  linear, 

while  the  effect  of  the  control  shows  up  only  in  the  rth  equation.  Thus,  by  defining  the  static  state 
feedback 

+  (2-51) 

with  V  an  arbitrary  reference  input,  we  end  up  with  a  closed  loop  system  governed  by  the  equations 

Zi  =:  Z2 
^2  =  ^3 


Zj'  -  \  -  Zrp 
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Zr  —V 

Zr+l  =  Qr+liz) 

Zn=qn{z)  (2.52) 

and  now  we  see  that  the  subsystem  consisting  of  the  first  r  equations  has  been  rendered  linear 
and  controllable  by  the  change  of  coordinates  in  Lemma  2.2.3  and  application  of  the  static  state 
feedback  defined  in  (2.51).  Thus,  if  our  system  had  relative  degree  r  =  n,  by  the  use  of  the 
above  transformation  and  state  feedback,  our  entire  set  of  dynamics  would  be  rendered  linear  and 
controllable,  accomplishing  our  stated  purpose  of  feedback  linearization.  Achieving  this  complete 
linearization  of  all  the  state  equations  is  known  as  solving  the  SISO  State  Space  Exact  Linearization 
Problem^  and  we  have  just  shown  that  a  sufficient  condition  for  solving  it  is  for  our  SISO  system  to 
have  relative  degree  r  =  n  at  some  point  of  interest.  It  turns  out  this  is  also  a  necessary  condition, 
as  formalized  in  the  following  theorem. 

Theorem  2.2.2  The  SISO  State  Space  Exact  Linearization  Problem  is  solvable  iff  3  a  neighborhood 
U  of  and  a  real-valued  function  A(aj),  defined  on  U,  such  that  the  SISO  system 

X  —  a(a^)  +  b{x)u  (2,53) 

y  -  A(r)  (2.54) 

has  relative  degree  n  at  x^ , 

Proof:  See  [33],  Lemma  4.2.1.  ■ 

As  can  be  seen  from  Theorem  2.2.2,  the  solvability  of  the  SISO  State  Space  Exact  Linearization 

Problem  requires  the  existence  of  a  suitable  output  function  for  which  the  system  has  relative  degree 
n.  It  turns  out  that  the  existence  of  such  a  function  can  be  tied  to  conditions  on  the  vector  fields  a 
and  h  in  (2.53).  These  conditions  are  specified  in  the  following  theorem. 
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Theorem  2.2.3  Suppose  a  SISO  system 

X  —  a{x)  +  h{x)u 

is  given.  The  SISO  State  Space  Exact  Linearization  Problem  is  solvable  near  a  point  (i.e.  3  an 
output  function  A(a:)  for  which  the  system  has  relative  degree  n  at  x^)  iff  the  following  conditions 
are  satisfied: 

L  the  matrix  adab{x^)  ...  ad'^~^b{x^)  ad2~^b{x^)]  has  rank  n 

a.  the  distribution  D  =  span{b,  adab, . .  ♦ ,  ad2~‘^b}  is  involutive  near 

Proof:  See  [33],  Lemma  4.2.2.  ■ 

Thus,  given  a  SISO  system  (2.53),  to  solve  the  SISO  State  Space  Exact  Linearization  Problem, 
Ave  first  check  conditions  (i)  and  (ii)  of  Theorem  2.2.3.  If  these  are  met,  we  next  seek  the  output 
function  X{x).  It  turns  out  that  one  can  solve  for  A  by  one  of  two  methods,  either  straightforwardly 
from  the  definition  of  relative  degree,  i.e.,  solve 

LtX{x)  =  LbLaX{x)  -  . . .  =  LtL2-^X{x)  -  0  V  a;  near  x^  (2.55) 

LtL2~^X{x^)  #  0  (2.56) 

or  equivalently,  solve 

LbX{x)  =  Lad^bHx)  =  . . .  =  =  0  V  a:  near  x°  (2.57) 

(2.58) 

We  then  choose  the  coordinate  transformation  $  according  to  Lemma  2.2.3,  i.e.,  set 

$(a:)  =  [A(a;)  L„A(a;)  ...  Lr'A(a;)]’' 
and  construct  the  linearizing  feedback 

u  =  (t>{x)  +  e{x)v  (2.59) 
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by  setting 


-L-M 

(2.60) 

(2.61) 

We  conclude  this  section  by  making  a  few  remarks  concerning  the  above  procedure.  First,  the 
requirement  of  relative  degree  n  assures  that  the  denominators  of  (2.60)  and  (2.61)  are  nonzero, 
and  thus  that  the  linearizing  feedback  is  well  defined.  Second,  more  than  one  appropriate  output 
function  A  may  exist,  as  solutions  to  (2.55),  (2.56),  (2.57),  and  (2.58)  are  not  unique.  Third,  we 
note  that  it  can  be  shown  that  condition  (i)  of  Theorem  2.2.3  is  a  type  of  controllability  condition. 
In  fact,  it  is  equivalent  to  the  requirement  that  the  linear  approximation  of  the  system  at  x  =  0  be 
controllable  [33]. 

The  above  procedure  is  the  one  to  follow  if  the  designer  is  free  to  choose  the  output  function 
A(ir).  If  given  an  output  function  c(x),  however,  one  simply  determines  the  corresponding  relative 
degree,  r,  of  the  system.  If  r  is  equal  to  n,  a  closed  loop  system  with  all  n  state  equations  linear  and 
controllable  is  obtained,  and  the  SISO  State  Space  Exact  Linearization  Problem  is  solved.  If  r  <  n, 
then  all  of  the  state  equations  will  not  be  rendered  linear  and  controllable  by  the  above  procedure. 
In  fact,  we  previously  established  that  only  r  of  the  closed  loop  state  equations  will  be  rendered 
linear  and  controllable.  We  discuss  the  case  where  r  <  n  in  the  next  section. 

2. 2. 2. 3  Input-Output  Linearization  and  Zero  Dynamics 

If  we  have  a  SISO  system  (2.53)  and  an  output  function  A(ir)  for  which  the  relative  degree  r  is 
strictly  less  than  n,  then  Equation  (2.52)  shows  that  if  we  follow  the  procedure  in  the  preceding 
section  (namely  coordinate  transformation  via  Lemma  2.2.3  and  state  feedback  via  (2.59),  (2.60), 
and  (2.61)  with  r  replacing  n),  we  obtain  a  closed  loop  system  such  that  the  first  r  equations  will  be 
linear  and  controllable,  and  the  remaining  n~  r  equations  will  be  nonlinear  and  unaffected  by  the 
control.  It  can  also  be  seen  that  the  input-output  behavior  is  totally  captured  by  the  first  r  state 
equations,  and  is  therefore  linear  and  controllable.  We  can  then  use  any  linear  design  technique  we 
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choose  to  ensure  that  this  r- dimensional  subsystem  of  the  state  dynamics  is  stable  and  well  behaved 
in  terms  of  performance.  Thus,  the  procedure  enables  us  to  obtain  the  desired  linear  and  controllable 
relationship  between  input  and  output,  but  this  solution  is  not  complete  because  the  last  n  —  r  state 
equations  are  uncontrollable  and  unobservable,  and  we  must  therefore  consider  their  stability  to 
ensure  the  overall  system  remains  stable.  We  proceed  to  address  this  issue  by  introducing  some 
simplifying  notation  and  then  invoking  some  previously  established  stabilization  concepts. 

We  shall  first  decompose  the  transformed  state  vector  into  two  groups:  the  first  r  state  variables 
and  the  last  n  —  r  of  them.  Thus,  let 


- 

Zr-\-l 

Zr 

Zn 

Then  the  normal  form  of  the  above  described  system  can  be  written  more  simply  as 


ii  =  Z2 


k  =  Zz 


ir-1  = 

=/?(^,>?)  +  a(^,»?)u  (2.62) 

»)  =  2(e,»7)  (2.63) 

Clearly,  if  we  design  the  control  to  ensure  stability  of  the  linearized  ^  subsystem  (stability  of  ^  =  0  in 
(2.62)),  and  if  we  can  establish  the  local  asymptotic  stability  of  77  =  0  in  (2.63),  by  methods  such  as 
those  of  Section  2.1,  this  will  be  sufficient  to  guarantee  the  local  asymptotic  stability  of  the  overall 
closed  loop  system.  It  is  not  necessary,  however,  as  the  lemmas  of  Section  2.1.3  show.  For,  recalling 
Lemma  2.1.4,  a  sufficient  condition  for  local  asymptotic  stability  of  the  origin  is  that,  if  ^  =  0  is 
LAS  and  if  the  zero  dynamics  of  the  system  are  LAS,  i.e.,  77  =  0  of 

?)  =  5(0,77)  (2.64) 
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is  LAS,  then  the  origin  (^,r?)  =  (0,0)  of  the  overall  system  is  LAS.  The  dynamics  of  (2.64)  are  called 
the  zero  dynamics  because  they  describe  the  evolution  of  the  state  trajectory  when  the  output 
y  is  constrained  to  equal  zero  for  all  time.  It  turns  out  that  analyzing  the  stability  of  (2.64)  is 
often  much  simpler  than  analyzing  that  of  (2.63),  and  thus  represents  a  significant  simplification  in 
required  analysis.  We  conclude  this  section  by  remarking  that  the  zero  dynamics  prove  critical  when 
the  linear  approximation  of  (2.64)  at  77  =  0  has  no  positive  eigenvalues,  else  we  could  invoke  the 
Principle  of  Stability  in  the  First  Approximation  to  conclude  instability  of  the  closed  loop  system. 

2. 2.2.4  Other  Interesting  Results 

In  this  section,  we  briefly  summarize  two  other  interesting  problems  and  results  involving  feedback 
linearization  of  SISO  systems:  asymptotic  output  tracking  and  the  disturbance  decoupling  problem. 
In  the  asymptotic  output  tracking  problem,  we  are  interested  in  producing  a  system  output,  which, 
irrespective  of  the  initial  system  state,  converges  asymptotically  to  a  prescribed  reference  output 
yfi[t).  It  turns  out  that  this  may  be  achieved  if  a  system  has  a  well  defined  relative  degree,  r,  in 
a  neighborhood  of  ?/h,  and  if  certain  other  conditions  are  met.  This  can  be  seen  from  recalling  the 
normal  form  equations,  (2.62),  (2.63),  for  a  system  with  relative  degree  r,  and  choosing  the  control 
input 

»  =  +  Sk’  -  £>^-1(21  - 

If  we  define  e{t)  =  y{t)  -  yR{t)  and  recall  y  —  obtain 

-  Cr-ie^’^~^^  —  ...  —  —  Coe  (2.66) 

or 

+  . . .  -h  cie^^^  +  Cqc  =  0  (2.67) 

so  that  if  the  are  chosen  to  ensure  that  all  the  roots  of  the  characteristic  equation  for  (2.67)  have 
negative  real  parts,  the  linearized  portion  of  the  normal  form  equations  will  exponentially  converge 
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to  the  desired  trajectory.  We  still  must  be  concerned  with  the  internal  dynamics  equations,  however, 
as  formalized  in  the  following  theorem. 

Theorem  2*2.4  Suppose  ,,,  defined  for  all  t  >  0  and  bounded^  and  let 

^R{t)  =  [yR{t)  •  ••  ^  Vr{^)  denote  the  solution  of 

y^qiUtU)  (2.68) 

satisfying  riR{0)  =  0.  Suppose  this  solution  is  defined  for  all  i  >  0,  bounded^  and  uniformly  asymp¬ 
totically  stable,  and  the  roots  of  the  characteristic  equation  for  (2.67)  all  have  negative  real  parts. 
Then  there  exists  a  neighborhood  of  the  total  reference  trajectory,  g^{t)Y }  such  that  for  all 

initial  conditions  located  in  this  neighborhood,  the  closed  loop  system  response  will  asymptotically 
converge  to  the  reference  trajectory. 

Proof:  See  [33],  Proposition  4.5.1.  ■ 

A  similar  but  more  elaborate  procedure  exists  for  achieving  asymptotic  model  matching  of  a 
linear  reference  model.  The  reader  is  referred  to  [33]  for  details. 

The  disturbance  decoupling  problem  involves  a  system  of  the  form 

X  =  a{x)  +  b{x)u  -H  g{x)d 

y  =  cix)  (2.69) 

where  d  is  an  undesired  input,  or  disturbance,  that  we  desire  not  to  affect  the  system  output  y  at 

all.  Once  again,  it  turns  out  that  under  certain  conditions  given  below,  this  may  be  achieved  using 
a  static  state  feedback,  and  that  the  system  having  some  relative  degree  r  at  a  point  of  interest  x^ 
provides  one  component  to  solving  this  problem,  as  formalized  below. 

Theorem  2*2,5  Suppose  the  SISO  system  (2.69)  has  some  relative  degree  r  at  a  point  of  interest 
x^ .  The  problem  of  finding  a  feedback  u  =  (j)[x)-\-d{x)v ,  defined  locally  around  x^,  with  v  an  arbitrary 
reference  input,  such  that  the  output  of  the  system  is  decoupled  from  the  disturbance  can  be  solved  iff 

LgLl^c{x)  =  0  0  <  i  <  r  -  X  near  x^  (2,70) 
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If  this  is  the  case^  one  solution  is 


^  =  + _ 

LbLl-^c(x)  LbLl-hix) 

Proof:  See  [33],  Proposition  4.6.1.  ■ 

2.2.3  Extensions  to  Multi-Input  Multi-Output  Systems 

In  this  section  we  present  an  extension  of  the  results  of  Section  2.2.2  to  the  full  multivariable  case 
[33].  Conceptually,  very  little  changes.  However,  we  must  amend  some  of  the  SISO  definitions  to 
make  sense  in  the  MIMO  setting.  Following  this,  we  present  the  multivariable  generalizations  of  the 
SISO  results  previously  shown. 

Recall  (2.32),  defining  a  nonlinear  dynamic  MIMO  system  with  m  affine  inputs  and  p  =  m 

outputs.  Note  that  we  have  assumed  a  square  system.  Such  a  system  is  said  to  have  (vector) 

relative  degree  {ri, . . . ,  at  a  point  if 

i-  V  1  <  j  <  m,  V  A:  <  ri  -  1,V  1  <  ^  <  m,  and  V  in  a  neighborhood  of 

Lb,Llciix)=0  (2.71) 

ii.  the  m  X  m  matrix 

Lb,Ll^-^c^ix)  •••  Lb^Ll^~^c^{x) 

Lb,Ll--'^C2{x)  •••  Lb^Ll-~'^C2{x) 

Lb^U'a”'-^Cm{x)  ■■■  Lb^Ll"'-'^Cm{x) 

is  nonsingulax  at  a;  =  x^. 

We  remark  that  this  definition  includes  the  definition  given  for  a  SISO  system  in  Section  2. 2. 2.1, 
and  that  vi  is  exactly  the  number  of  times  one  must  differentiate  the  ith  output  at  t  in  order 
to  have  at  least  one  component  of  the  input  vector  u{t^)  explicitly  appearing  in  the  right  hand  side 
of  the  expression.  With  this  definition,  Lemma  2.2.3  of  Section  2. 2.2.1  extends  to  the  MIMO  case 
in  a  straightforward  manner  as  given  below. 
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Lemma  2.2.4  Suppose  the  system  (2.32)  with  m  =  p  has  (vector)  relative  degree  at 

x°.  Then  r  =  ri  +  . . .  +  rm  Set,  for  1  <  i  <m, 

=  Ci{x) 

(t>\{x)  =  LaCi{x) 


4^ix)  =  Ll*-^Ci{x)  (2.73) 

Ifr  <  n,3  n  —  r  more  functions  ^^+1(2:), . . . ,  (t)n{x)  such  that  the  mapping 

•••  <l)l,{x)  ...  4>f{x)  ...  (j>f^{x)  4)r+i{x)  ...  0„(a:)]^ 

has  a  nonsingular  Jacobian  matrix  at  ,  The  value  of  these  additional  functions  at  can  be  fixed 
arbitrarily.  Moreover,  if  the  distribution 

D-  span{bi, . . .  ,hm} 

is  involutive  near  x^  it  is  always  possible  to  choose  <j)r.\-i{x), . . . ,  <j)n{'^)  such  that 

Lbj(fi{x)  =  0  V  r  -hi  <  i  <  n,V  1  <  i  <  m,V  x  near  x^  (2.74) 

Again  from  Theorem  2.2.1,  we  see  that  $  in  Lemma  2.2.4  qualifies  as  a  local  diffeomorphisin.  The 
only  thing  new  in  Lemma  2.2.4  is  the  requirement  for  the  distribution  D  to  be  involutive  in  order  to  be 
able  to  satisfy  (2.74).  This  is  not  really  a  new  requirement,  however,  as  it  was  mentioned  previously 
that  any  1-dimensional  distribution  is  involutive,  and  so  this  requirement  is  automatically  satisfied 
in  the  SISO  case.  We  also  note  here  that  although  we  have  assumed  a  square  system,  these  results 
can  be  extended  to  nonsquare  systems  as  long  as  condition  ii  in  the  definition  of  (vector)  relative 
degree  is  replaced  by  the  assumption  that  the  matrix  T{x)  has  full  row  rank  at  [33]. 

With  the  coordinate  tranformation  defined  above,  we  now  revisit  the  concept  of  normal  forms, 
explicitly  characterizing  those  we  will  obtain  in  the  MIMO  case.  Consistent  with  previous  notation. 
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we  first  decompose  the  transformed  state  vector  by  letting 


for  1  <  2  <  and  set 


Also  let 


and 


01 W 

.  . 

0n  (^) 

Vi 

(jir+l  (x) 

Vn-r 

(i>n(x) 

=  l<i<m 

Then  the  normal  form  of  the  above  described  system  for  1  <  ^  <  m  can  be  written  more  simply  as 


zn 

Sri— 1 

m 

C(  =  ’j) + X] 

j=i 

If  (2,74)  is  satisfied,  then  the  final  n-r  state  space  equations  in  the  new  set  of  coordinates  is  of  the 
form 

V  =  <}{(,  v)  (2-76) 

while  if  (2,74)  is  not  satisfied  but  we  still  have  a  valid  coordinate  transformation,  we  will  have 


V  =  g{^,v)  +  p{^,v)u 


(2.77) 
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With  (2.75)  in  hand,  we  can  now  very  easily  give  a  sufficient  condition  for  and  a  solution  to  the 
State  Space  Exact  Linearization  Problem  for  MIMO  nonlinear  systems  of  the  form  (2.32),  If  we 
assume  (2.32)  has  a  (vector)  relative  degree  {ri, . . .  at  a  point  and  r  —  ri  -f  . . .  +  equals 
the  dimension  of  the  state  space  n,  then  since  the  matrix  T{x)  is  nonsingular  in  a  neighborhood  of 
a:®,  we  can  define  the  static  state  feedback 

u  =  T-\0[-m  +  ^]  (2-78) 

with  V  an  arbitrary  m- vector  to  yield  a  closed  loop  system  governed  by  the  m  sets  of  equations 

H  —  ci 

Sri  — 1 

=  Vi  (2.79) 

which  are  clearly  linear  and  controllable.  It  turns  out  that  the  above  condition  on  (vector)  relative 
degree  is  also  necessary,  as  in  the  SISO  case,  as  long  as  the  rank  of  h{x^)  is  m.  We  now  present  the 
MIMO  versions  of  Theorems  2.2.2  and  2.2.3. 

Theorem  2.2.6  Suppose  the  matrix  h{x^)  has  rank  m.  Then,  the  MIMO  State  Space  Exact  Lin¬ 
earization  Problem  is  solvable  iff^  a  neighborhood  U  ofx^  and  m  real-valued  functions  ci  (a^), . . . ,  Cm{x), 
defined  on  U,  such  that  the  system 

X  =  a{x)  +  h{x)n  (2.80) 

y  =  c{x)  (2.81) 

has  some  (vector)  relative  degree  {r\, . . .  .rm}  at  and  rj  +  r2  +  . . .  +  ”  n. 

Proof:  See  [33],  Lemma  5.2.1.  ■ 

We  remark  that  the  rank  m  requirement  for  b{x^)  can  be  relaxed,  and  refer  the  reader  to  [33]  for 
the  details  of  how  to  accomplish  this. 
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Theorem  2.2.7  Suppose  the  matrix  b{x^)  has  rank  m.  Then,  3  a  neighborhood  U  of  and  m 
real-valued  functions  Ai(a:;), . . . ,  A^(a;),  defined  on  U,  such  that  the  system 


X  —  a{x)  +  b{x)u 

(2.82) 

y  =  A(a;) 

(2.83) 

has  some  (vector)  relative  degree  {ri, . . .  ,rm}  at  with  fi  -f  r2  +  . . .  +  =  n,  iff: 

i  for  each  0  <  i  <  n  -  1,  the  distribution  Di  has  constant  dimension  near  where 

Di  —  span{ad^bj  :  0  <  k  <  i,l  <  j  <  m] 

ii.  the  distribution  Dn-i  has  dimension  n 
UL  for  each  0  <  i  <  n  —  2,  the  distribution  Di  is  involutive 

Proof:  See  [33],  Theorem  5.2.3.  ■ 

Thus  the  MIMO  State  Space  Exact  Linearization  Problem  is  solvable  iff  the  three  conditions  in 
Theorem  2.2.7  are  met.  For  a  technique  for  constructing  the  appropriate  output  functions  Xffx), 
the  reader  is  referred  to  [33]. 

Finally,  as  in  Section  2.2.2,  we  need  to  consider  the  nonexact  case,  i.e.,  the  case  in  which  the 
input-output  behavior  may  be  linearized,  but  not  all  n  of  the  transformed  state  equations  are 
rendered  linear.  Prom  our  previous  discussion  of  the  MIMO  normal  form,  (2.75),  it  is  clear  that  if 
a  system  (2.32)  possesses  some  vector  relative  degree  at  a  point  x^ ,  then  the  standard  coordinate 
transformation  and  state  feedback  may  be  applied  to  yield  an  r  =  ri  -j-. .  .-hr^T^ -dimensional  subsystem 
that  is  linear  and  controllable,  and  if  (2.74)  is  satisfied,  an  n-r  dimensional  subsystem  unaffected  by 
the  control.  As  in  Section  2.2. 2.3,  it  is  straightforward  to  apply  Lemma  2.1.4  to  conclude  that  if  the 
zero  dynamics  of  (2.75)  are  locally  asymptotically  stable,  then  the  overall  system  may  be  rendered 
LAS  by  appropriate  design  of  a  stabilizing  feedback  control  on  the  linearized  portion  of  the  state 
dynamics.  It  turns  out  that  although  having  a  vector  relative  degree  is  sufficient  to  obtain  linearized 
input-output  behavior  in  the  MIMO  case,  it  is  not  necessary  as  it  was  in  the  SISO  case.  In  fact,  a 
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broader  class  of  MIMO  systems  may  be  input-output  feedback  linearized  than  just  those  possessing 
a  vector  relative  degree.  These  systems  are  those  for  which  Silverman’s  Structure  Algorithm  [60] 
may  be  successfully  applied.  This  algorithm  is  rather  complicated  and  will  not  be  presented  here, 
but  the  reader  is  referred  to  [33]  for  complete  details. 

In  addition  to  those  discussed  for  the  SISO  case,  there  are  many  other  interesting  results  and 
problems  unique  to  MIMO  systems  described  by  (2.32),  only  three  of  which  will  briefly  be  discussed 
here.  It  turns  out  that  if  a  square  (m  x  m)  system  (2.32)  possesses  a  vector  relative  degree  at  a 
point  2?^,  this  is  sulBcient  to  guarantee  existence  of  a  solution  to  the  so-called  Noninteracting  Control 
Problem  [33] .  This  problem  is  defined  as  achieving  a  closed  loop  system  such  that  the  existence  of  a 
vector  relative  degree  is  preserved  and  each  control  input  affects  one  and  only  one  separate  output, 
i.e.,  the  closed  loop  system  is  decomposed  into  m  SISO  systems.  In  fact,  it  is  easy  to  see  that  the 
feedback  control  defined  by  (2.78)  achieves  the  desired  purpose. 

From  all  the  results  presented  thus  far,  it  is  clear  that  the  existence  of  a  vector  relative  degree 
for  a  system  (2.32)  is  very  useful  for  control  design.  When  a  given  system  does  not  possess  such 
a  relative  degree,  it  may  be  possible  to  achieve  one  by  augmenting  the  system  with  integrators  on 
certain  of  the  system  inputs.  This  procedure  is  often  called  achieving  relative  degree  via  Dynamic 
Extension^  and  an  algorithm  may  be  found  in  [33].  A  point  of  interest  is  that  by  this  state  variable 
augmentation,  this  procedure  will  result  in  a  dynamic  nonlinear  controller,  whereas  up  to  this  point, 
all  other  state  feedback  controllers  considered  have  been  static  (nonlinear)  feedbacks. 

The  final  result  we  wish  to  present  is  not  unique  to  the  theory  of  feedback  linearization,  but 
instead  is  useful  in  all  cases  where  we  wish  to  achieve  local  asymptotic  stabilization  of  a  MIMO 
system  such  as  (2.32).  This  result,  taken  from  [33],  relates  stabilizability  of  the  linear  approximation 
of  (2.32)  at  the  origin 

X  =  Ax  -h  Bu  (2.84) 

where  A  equals  the  Jacobian  of  a  at  0  and  B  =  &(0),  to  the  stabilizability  of  the  full  nonlinear 
system,  using  either  linear  or  nonlinear  feedback. 
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Theorem  2.2.8  Suppose  the  linear  approximation  (A^B)  of  (2.84)  stahilizable.  Then  any  linear 
feedback  which  asymptotically  stabilizes  the  linear  approximation  also  asymptotically  stabilizes  the 
original  nonlinear  system,  at  least  locally.  If  the  pair  (A,  B)  is  not  controllable  and  there  exist 
uncontrollable  modes  associated  with  eigenvalues  with  positive  real  part,  then  the  original  nonlinear 
system  cannot  be  stabilized  at  all 

Proof:  See  [33],  Proposition  4.41.  ■ 

Theorem  2.2.8  is  easily  proven  by  invoking  the  Principle  of  Stability  in  the  First  Approximation. 
The  interesting  aspect  of  the  theorem  is  that  stabilizability  of  the  linearization  is  sufficient  but  not 
necessary  for  local  asymptotic  stabilization  of  a  nonlinear  system.  Indeed,  Theorem  2.2.8  implies 
that  a  nonlinear  dynamic  system  whose  linearization  is  not  stahilizable  may  still  be  stahilizable  in 
the  nonlinear  sense,  provided  any  uncontrollable  modes  of  the  linearization  have  eigenvalues  with 
zero  real  part.  We  will  consider  stabilization  of  just  such  a  critical  system  in  Chapter  3. 

2.3  Recursive  Backstepping 

In  this  section  we  present  the  basic  theory  for  nonlinear  control  design  via  the  technique  of  recursive 
backstepping.  Similar  to  Section  2.2,  we  start  first  with  single-input  systems,  and  then  proceed  to 
the  multiple-input  case.  In  fact,  virtually  all  the  existing  results  are  for  single-input  systems.  The 
great  majority  of  the  material  is  taken  from  [40] . 

2.3.1  Theory  for  Single-Input  Systems 

In  Section  2.1.1,  we  introduced  the  concept  of  a  Lyapunov  function  for  a  system  governed  by  the 
vector  differential  equation 

X  =  a{x,  t) 

The  main  idea  was  that  the  existence  of  a  Lyapunov  function  was  sufficient  to  guarantee  at  least  the 
local  stability  of  an  equilibrium  point  of  the  system.  Such  a  formulation  is  useful  if  one  completely 
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knows  the  drift  function  a,  and  is  simply  seeking  to  determine  system  stability.  If,  however,  we  now 
consider  the  time-invariant  system 

X  =  fix,  u),  xen^,uen,  /(0, 0)  =  0  (2.85) 

where  we  are  free  to  choose  a  feedback  control  input  u  =  a{x),  a  natural  question  to  ask  is,  does 
there  exist  a  u  such  that  the  resulting  closed  loop  system  is  globally  asymptotically  stable?  If  the 
answer  is  yes,  the  system  (2.85)  is  said  to  possess  a  control  Lyapunov  function  (CLF).  Artstein 
[2]  established  a  necessary  and  sufficient  condition  for  the  existence  of  a  CLF  for  a  system  of  the 
form  of  (2.85),  presented  below. 

Lemma  2.3.1  A  smooth,  positive  definite,  and  radially  unbounded  function  V  :  'R.'^  — »  is  a 

control  Lyapunov  function  for  (2.85)  if 

ill  <  0  v*  ^  0  (2.86) 

This  intuitively  simple  concept  of  finding  a  feedback  control  and  associated  Lyapunov  function  for 
the  resulting  closed  loop  system  equations  forms  the  basis  of  control  Lyapunov  function  design 
theory.  The  problem  is  that  (2.86)  speaks  to  the  existence  of  the  pair  u  and  V,  but  provides  little 
insight  into  how  to  choose  them  simultaneously  to  guarantee  that  the  closed  loop  system  is  GAS. 
Recursive  backstepping  is  one  attempt  to  remedy  this  problem.  It  provides  a  systematic  yet  flexible 
procedure  for  sequentially  building  up  the  feedback  control  and  CLF  to  accomplish  the  desired 
stability  objective.  Actually,  recursive  backstepping  requires  slightly  different  assumptions  for  a 
system  than  does  control  Lyapunov  function  theory.  In  fact,  for  an  affine  single-input  system,  it 
requires  the  following  assumption. 

Assumption  2.3.1  Consider  the  system 

X  =  a(a,’)  -t-  b{x)u,  a(0)  =  0  (2.87) 

where  x  eTl^  is  the  state  and  u  e  R  is  the  control  input.  There  exist  a  continuously  differentiable 
feedback  law 

u  =  ot(x),  a(0)  =  0  (2.88) 
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and  a  smooth,  positive  definite,  radially  unbounded  function  V  :  7Z^  such  that 

^{x)[a{x)  +  &(a:)a(a;)]  <  -W{x)  <  0,  V  a;  €  7^”  (2.89) 

where  W  :  71'^  is  positive  semidefinite. 

Under  this  assumption,  the  control  (2.88),  applied  to  the  system  (2.87),  guarantees  via  the  Lasalle- 
Yoshizawa  theorem  (Theorem  2.1.2)  the  global  boundedness  of  x{t)  and  the  asymptotic  regulation 
of  W{x{t)).  Lasalle’s  Invariance  theorem  (Theorem  2,1.4)  with  0  =  may  also  be  applied  to 
conclude  that  x{t)  converges  to  the  largest  invariant  set  M  contained  in  the  set  E  —  {x  £  \ 

W{x)  =  0}.  If  we  take  W{x)  in  (2.89)  to  be  positive  definite  instead  of  positive  semidefinite  (as  is 
the  assumption  for  CLF  theory),  then  clearly  the  control  (2.88)  renders  a;  =  0  the  GAS  equilibrium 
of  (2.87).  Besides  the  different  assumptions  on  the  definiteness  of  W (x),  the  other  difference  between 
recursive  backs tepping  and  CLF  theory  is  that  we  have  required  the  control  (2.88)  to  be  continuously 
differentiable  in  Assumption  2.3.1,  whereas  there  is  no  such  requirement  in  CLF  theory.  This 
property  of  the  control  is  crucial  to  the  recursive  backstepping  design  procedure,  as  illustrated  in 
the  following  basic  theorem. 

Theorem  2^3*1  (Integrator  Backstepping)  Let  the  system  (2.87)  be  augmented  by  an  integrator 
on  the  control  input: 

X  =  a{x)  -f  6(a;)^  (2.90) 

i  =  u  (2.91) 

and  suppose  that  (2.90)  satisfies  Assumption  2.3.1  with  ^  G  IZ  as  its  control. 

i.  IfW(x)  is  positive  definite,  then 

Vaix,0  =  V{x)  +  ^[^-aix)r  (2.92) 

is  a  CLF  for  the  full  system  (2.90),  (2,91).  Thus,  3  a  feedback  control  u  =  aa{x,^)  which 
renders  (a;,  ^)  =  (0,0)  the  GAS  equilibrium  of  the  system.  One  such  control  is 

da  dV 

u  =  -c(e  -  a{x))  +  ^(a;)[a(x-)  +  6(a:)C]  -  c  >  0  (2.93) 
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a,  IfW{x)  is  only  positive  semidefinite,  then3  a  feedback  control  which  renders  Va  <  —Wa{x,^)  < 
0,  such  that  Wa{x^  C)  >  0  whenever  W{x)  >  0  or  ^  7^  a{x).  This  guarantees  global  boundedness 
and  convergence  of  [x'^{t)  largest  invariant  set  Ma  contained  in  the  set  Ea  = 

{[x'^  f]^  e  I  W{x)  =  0,  e  =  Ci{x)}, 

The  proof  of  Theorem  2.3.1  is  very  instructive  of  the  procedure  in  general  and  allows  us  to  define 
some  necessary  terms.  We  therefore  include  it  in  its  entirety  [40]. 

Proof:  Examining  Equation  (2.90),  we  see  that  ^  now  plays  the  role  that  u  did  in  Assumption 

2.3.1.  We  therefore  consider  ^  to  be  a  virtual  control  for  (2.90),  and  we  choose  a  desired  feedback 
control  function  a(x)  that  will  result  in  closed  loop  stability  for  (2.90)  if  we  are  able  to  make 
(  -  01(2;)  by  appropriately  choosing  u  in  (2.91).  Accordingly,  ck(x)  is  called  a  stabilizing  function. 
We  recognize,  however,  that  we  will  never  achieve  perfect  equality,  and  so  we  introduce  the  error 
variable 

e  =  ^  -  a{x)  (2.94) 

to  aid  in  the  analysis.  Differentiating  (2.94)  with  respect  to  time,  we  can  rewrite  the  system  equations 
as 


X  —  a{x)  -f  h{x)[a{x)  +  e] 


e  —  u~ 


—  {x)[a(x)  -h  6(a:)(a(2;)  -h  e)] 


(2.95) 


Using  (2.89),  the  derivative  of  (2.92)  along  the  solutions  of  (2.95)  is 


dV 

Va  =  (a  -f  &Q:  +  6e)  -f  e 


w  -  +  Kot  +  e)) 


dV , 

=  -^{a+ba)+e 


< -W{x)+e 


'  9a,  ,,  dV^ 

u  —  -^{0,  +  b{a  +  e))  +  -^b 


dx 

da,  ,  ..  dV 

—{a  +  b(a  +  e))+—b 


dx 


(2.96) 


where  the  terms  containing  e  have  been  grouped  together.  By  the  Lasalle-Yoshizawa  theorem,  any 
choice  of  the  control  u  which  renders  Va  <  — lVa(a;,  f)  <  —W{x),  with  Wa  positive  definite  in  e, 
guarantees  global  boundedness  of  x,e,  and  and  regulation  of  W{x{t))  and  e{t).  Furthermore, 
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Lasalle’s  Invariance  theorem  guarantees  convergence  of  [x'^{t)  to  the  largest  invariant  set 

contained  in  the  set  {[x'^  e]'^  e  |  =  0,  e  O}.  An  obvious  way  to  make  14  negative 

definite  in  e  is  to  choose  the  control  as  in  (2.93),  which  renders  the  braced  term  in  (2.96)  equal  to 
~ce  and  yields 

14  <  ~W{x)  -  ce^  =  -Wa(x,C)  <  -W{x)  <  0  (2.97) 

Clearly,  if  W{x)  is  positive  definite,  Theorem  2.1.2  guarantees  the  global  asymptotic  stability  of 
(x,  e)  —  (0,0),  which  in  turn  implies  that  Vo(x,  ^)  is  a  CLP  and  (x,  ^)  =  (0,0)  is  the  GAS  equilibrium 
of  (2.95).  ■ 

While  the  choice  of  control  (2.93)  is  simple,  it  is  not  unique  and  may  not  even  be  the  best 
choice  when  considering  other  factors  such  as  control  usage,  because  it  cancels  all  the  nonlinearities 
in  the  braced  term  in  (2.96),  some  of  which  may  be  beneficial  to  stability  or  dominated  by  more 
powerful  terms,  and  thus  not  require  cancellation.  The  authors  of  [40]  state  that  the  main  result 
of  backstepping  is  not  a  specific  control  law,  but  the  construction  of  a  Lyapunov  function  whose 
derivative  can  be  made  negative  definite  by  a  variety  of  control  laws.  The  designer  thus  has  significant 
freedom  when  using  this  method,  and  must  exercise  good  engineering  judgment  when  selecting  the 
control. 

Although  Theorem  2.3.1  shows  only  one  integrator,  if  the  original  control  u  in  (2.87)  is  k  times 
continuously  differentiable,  it  is  straightforward  to  extend  the  method  of  integrator  backstepping 
to  a  chain  of  k  augmented  integrators,  with  the  associated  introduction  of  k  stabilizing  functions 
selected  sequentially,  starting  from  the  original  equation,  and  k  error  variables,  quadratic  positive 
definite  terms  of  which  are  added  to  the  original  V(x)  to  construct  the  overall  system  Lyapunov 
function.  While  integrator  backstepping  can  be  quite  useful  for  design,  a  more  general  technique 
may  be  needed.  The  next  theorem  provides  the  necessary  generality. 

Theorem  2.3.2  (Nonlinear  Block  Backstepping)  Consider  the  cascade  system: 

X  —  a{x)  -I-  h{x)y  (2.98) 

e  =  m(x,  0  +  /3(x,  0^,  y  =  c(e),  c(0)  -  0,  ^  G  ^  e  7^  (2.99) 
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Assume  that  (2,99)  has  globally  defined  and  constant  relative  degree  one  uniformly  in  x,  and  that 
its  zero  dynamics  subsystem  is  ISS  with  respect  to  x  and  y  as  inputs.  Also  suppose  that  (2,98) 
satisfies  Assumption  2.3,1  with  y  as  its  control,  except  V{x)  may  be  only  positive  semidefinite,  and 
we  require  the  closed  loop  system  solution  x{t)  to  be  bounded  if  V{x{t))  is  bounded.  Then  there 
exists  a  feedback  control  which  guarantees  global  boundedness  and  convergence  of  [x^ (t)  to 

the  largest  invariant  set  Ma  contained  in  the  set  Ea  ~  {[2^^  ^  |  \Y{x)  —  0,  1/  =  a(a;)}. 

One  such  control  is 

{fir  dV  1 

-c,{y  -  a{x))  -  >  »  (2-100) 

Moreover,  if  both  F(a:)  and  W{x)  are  positive  definite,  then  the  equilibrium  (a:,  =  (0,0)  is  GAS. 

Proof:  See  [40],  Lemma  2.25.  ■ 

2.3.2  Theory  for  Multi-Input  Systems 

The  entire  recursive  backstepping  theory  for  multi-input  systems  to  be  found  in  [40]  consists  of  a 
single  design  procedure  for  systems  in  what  the  authors  call  multi-input  parametric  strict-feedback 
form.  This  procedure  includes  provisions  for  parametric  uncertainty  in  the  system  equations,  which 
we  will  not  cover  here.  In  what  follows,  this  author  has  therefore  adapted  the  procedure  to  deal  with 
nominal  control  design  only,  which  applies  for  systems  in  the  resulting  multi-input  strict-feedback 
form: 

x{  =  x\ 

Ki-i  =< 

m 

(2-101) 

a=i 

where  1  <  i  <  m,  0^®  added  condition  that  the  control  input  matrix  T 

is  nonsingular  for  all  x  e  7?.".  Upon  comparing  (2.101)  with  (2.75),  we  see  that  apart  from  there 
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being  no  drift  term  P{x)  in  the  last  equation  of  (2.101),  the  two  sets  of  equations  are  identical,  as 
long  diS  Y^Vi  ==  n  in  (2.75)  as  well.  Thus,  the  multi-input  strict-feedback  form  can  be  obtained  for 
any  system  possessing  a  (vector)  relative  degree  such  that  fhe  system 

is  exactly  state  space  feedback  linearizable.  The  multi-input  strict-feedback  recursive  backstepping 
procedure  for  a  system  with  m  inputs  is  given  in  the  following  theorem. 

Theorem  2.3.3  (Multi- Input  Strict-Feedback  Recursive  Backstepping)  For  the  ith  subsys¬ 
tem  of  (2A01  )j  1  <  i  <  m,  define  for  1  <  k  <ri  the  error  variables, 

—  *  5  ^l-l)  (2.102) 


and  the  stabilizing  functions 


k-l 


dof' 

=  -44  -  4-1  +  Y. 

where  ef^  =  a q  =  0,  and  >  0  are  design  constants.  Also,  let  the  control  input  be  given  by 


(2.103) 


(2.104) 


Then  x{t)  and  e(t)  are  globally  bounded  and  e{t)  0  as  t  ^  cc.  Furthermore,  x  —  0  is  a  GAS 
equilibrium  of  the  closed  loop  system. 


Proof:  See  [40],  Theorem  3.5  and  Section  3.3.2. 


2.4  Nonlinear  Hqq  Control 

In  this  section  we  present  the  modern  theory  of  nonlinear  Hqo  control  design,  perhaps  more  appropri¬ 
ately  called  nonlinear  induced  L2  control  design.  Our  treatment  begins  with  the  problem  definition 
and  required  assumptions,  and  then  we  briefly  discuss  some  concepts  related  to  the  notion  of  in¬ 
duced  L2  gain  for  nonlinear  systems.  Following  that,  we  present  sufficient  conditions  for  achieving 
the  desired  control  objective  under  both  full  state  feedback  and  output  feedback  assumptions.  The 
majority  of  this  material  can  be  found  in  [34]  and  [67]. 
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2,^.1  Problem  Definition 


In  this  section  we  consider  general  nonlinear  systems  of  the  form 

X  ™  f{x,d,  u) 
y  =  c{x,d) 

z  =  h{x,u)  (2.105) 

with  state  vector  x  €  exogenous  input  d  e  71'^,  control  input  u  €  measurement  vector 
y  e  71^^  and  penalized  variable  z  e  We  note  that  d  may  contain  both  disturbances  to  be 
rejected  (process  and  measurement  noises)  and  reference  signals  to  be  tracked,  and  that  may 
contain  both  tracking  type  errors  and  penalties  on  control  usage.  We  assume  /,  h,  and  c  to  be 
smooth  vector  valued  functions  defined  in  a  neighborhood  of  the  origin  in  71^  x  7Z^  We  also 

assume  the  existence  of  a  fixed  equilibrium  for  (2.105),  and,  as  discussed  in  Section  2.1.1,  we  may 
without  loss  of  generality  take  it  to  be  the  origin  so  that  /(0,0, 0)  =  0,  h{0, 0)  =  0,  and  c(0, 0)  ==  0. 
Now,  let  7  be  a  fixed  positive  constant.  Then  the  nonlinear  ifoo  suboptimal  control  problem  (for 
disturbance  attenuation  level  7)  is  to  find  a  nonlinear  (dynamic)  compensator 

e  =  Mc,2/) 

u-ni{^,y)  (2.106) 

with  state  vector  and  k  and  m  smooth  functions  satisfying  A;(0, 0)  =  0,  m(0,0)  —  0,  such  that 
the  closed  loop  system  resulting  from  (2.105)  and  (2.106)  has  induced  L2  gain  less  than  or  equal  to 
7,  where  the  definition  of  induced  L2  gain  for  a  nonlinear  system  is  as  follows.  A  system  (2.105), 
(2.106),  has  induced  L2  gain  less  than  or  equal  to  7  if  for  all  initial  conditions  aj(0),  z{0)^  there  exists 
a  nonnegative  constant  K  (depending  on  a:(0),;^(0)  and  equal  to  zero  for  x{0)  —  0,^(0)  =  0)  such 
that 

r\\z{t)fdt<^^  r  \\d{t)\\‘^dt+K  (2.107) 

Jo  Jo 

for  all  d  €  T2[0,  T]  and  T  >  0,  with  z(t)  denoting  the  response  for  initial  condition  a:(0),  f  (0).  Then 
the  nonlinear  Hoo  optimal  control  problem  is  to  find  the  smallest  7*  >  0  such  that  the  nonlinear  Hqo 
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suboptimal  control  problem  is  solvable  for  all  7  >  7*.  In  the  next  section  we  provide  a  connection 
between  the  theory  of  dissipative  systems  and  the  induced  L2  gain,  which  will  allow  us  to  derive 
desirable  stability  properties  for  systems  with  finite  induced  L2  gain. 


24-2  Induced  L2  Gain  Analysis  and  Dissipativity  for  Nonlinear  Systems 


Following  standard  convention  [7,  27,  67],  a  nonlinear  system 


X  =:  f{x,  d) 

z  =  h{x)  (2.108) 


is  said  to  be  locally  dissipative  near  {x,  d)  =  (0, 0)  with  respect  to  the  supply  rate  s{d,  z)  if  there 
exists  a  positive  semidefinite  function  V{x)  such  that  for  all  x^uin  a.  neighborhood,  U,  of  (0, 0),  the 
dissipation  inequality 

V{x{T;  x{0),  0))  <  V{x(0))  +  [  s{dit),z{t))dt  (2.109) 

Jo 

holds  V  T  >  0.  V  above  is  called  a  storage  function  for  (2.108).  A  system  is  said  to  be  dissipative 
if  (2.109)  holds  and  U  ~7l^  x  TV.  If  we  let  the  supply  rate  s  —  7^||d|p  -  ||j2;||^,  we  see  (recalling  V 
is  positive  semidefinite)  that  (2.109)  becomes 

0  <  V{x{T;x{0),0))  <  V{x(0))  +  (2.110) 

Jo 


which  implies 

r\\z{t)fdt<  r  j^d(t)f +v{x{o))  (2.111) 

Jo  Jo 

Thus,  a  system  (2.108)  which  is  (locally)  dissipative  with  respect  to  the  supply  rate  7^||d|p  —  \\z\\^ 
has  (locally)  induced  L2  gain  less  than  or  equal  to  7. 

If  the  storage  function  y  is  (2.109)  is  equivalent  to  the  differential  dissipation  inequality 


^f{x,d)  -  s{d,z)  <  0 


(2.112) 


or,  again  letting  $  =  7^||d|p  —  \\z\\^ 


W 

dx 


f{x,d)  +  \\zf-'r^df<o 


(2.113) 
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A  consequence  of  (2.113)  is  that  for  d  —  0,  we  get 

^f{x,0)<-\\zr<0  (2.114) 

SO  that  if  the  storage  function  V  is  positive  definite,  it  serves  as  a  Lyapunov  function,  and  we  obtain 
(local)  asymptotic  stability  of  the  origin  if  the  inequality  in  (2.114)  is  satisfied  strictly.  Alternatively, 
it  turns  out  that  V  is  indeed  positive  definite  if  (2.108)  is  assumed  to  be  zero-state  observable, 
i.e.,  z{t)  —  0  y  t  >  0  ^  a;(0)  =  0  [67].  Moreover,  if  V  is  proper,  i.e.,  for  each  c  >  0  the  set 
{a:  €  72^"^  I  0  <  y  <  c}  is  compact,  then  the  origin  is  GAS.  If  V  is  only  positive  semidefinite,  then 
we  still  may  be  able  to  apply  Theorem  2.1.4  to  deduce  desirable  stability  properties  for  (2.108),  if 
the  system  meets  certain  requirements.  One  such  case  is  that  of  a  zero-state  detectable  system, 
which  is  one  for  which  z(t)  —  Q  \f  t  >  0  =>  x{t)  ^  0  as  t  »  oo  [34]. 

We  close  this  section  with  a  final  comment  relating  the  induced  L2  gain  of  an  input-affine  non¬ 
linear  system 

X  =  a{x)  4-  g{x)d^  a(0)  =  0 

2:  =  h{x),  h{0)  =  0  (2.115) 

to  that  of  its  linearization  at  zero 

X  ^  Ax  Gd 

z  =  Hx  (2.116) 

where  A  ~  (0),  G  —  51(0),  and  H  —  f”(0).  Van  der  Schaft  has  shown  [66]  that  if  A  is  asymptoti¬ 

cally  stable  and  (2.116)  has  induced  L2  gain  <  7,  then  so  does  (2.115)  locally,  and  vice  versa.  Now, 
the  induced  L2  gain  of  (2.116)  is  easily  computable  [56]  from  the  bounded  real  lemma  algebraic 
Riccati  equation 

A'^P  +  PA->r^PGG'^P  +  H'^H  =  Q  (2.117) 

by  finding  the  smallest  7  such  that  (2.117)  has  a  symmetric,  positive  semidefinite  solution  P  with 
the  eigenvalues  o{A-\-^GG'^P  all  having  negative  real  parts.  Thus,  this  result  offers  an  easy  way  to 
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bound  the  local  induced  L2  gain  of  (2.115),  and  provides  a  check  for  any  algorithm  we  may  compose 
to  estimate  the  optimal  induced  L2  gain  of  a  nonlinear  system. 


24 ’3  State  Feedback 

In  this  section  we  consider  the  nonlinear  Hoo  suboptimal  control  problem  under  the  assumption  of 
full  state  feedback.  Thus,  we  seek  a  feedback  law  of  the  form  u  ~  a{x)  such  that  the  closed  loop 
system 

X  ”  f{x,d,  a{x)) 

z  —  h{x,  (2.118) 

is  (locally)  dissipative  with  respect  to  the  supply  rate  s{d,z)  =  7^||d|p  —  \\z\\^.  This  problem  may 
be  cast  as  a  two-player,  zero-sum,  differential  game,  in  which  the  minimizing  player  controls  the 
input  u  and  the  maximizing  player  controls  the  exogenous  input  d  [4].  We  associate  with  this  game 
a  Hamiltonian  function  H  :  x  x  TV'  x  TV^  — »  TZ  defined  as 

n(x,  p,  d,  u)  =  p^f(x,  d,  u)  +  ||fe(a:,  u)f  -  'r^df  (2.119) 

If  we  now  assume  the  plant  (2.118)  satisfies  the  following  hypothesis: 

Assumption  2.4.1  The  penalty  map  h{x,u)  is  such  that  the  matrix 

p.  =  g(0,0) 

has  rank  m 


i.e.,  we  place  a  nontrivial  independent  linear  penalty  on  every  control,  then  the  Hessian  matrix  of 


W,  viewed  as  a  function  of  (d,  u),  at  (x’,p,  d,  u)  =  (0, 0, 0, 0)  equals 

-2j^I  0 

(2.120) 

0  2DfDi 

Thus,  the  Hamiltonian  H{Xj  p,  d,  u)  is  quadratic  in  d  and  u,  and  since  by  Assumption  2.4.1,  Dj D\  is 


positive  definite,  (2.120)  implies  H  has  a  unique  local  saddle  point  in  (d,  u)  for  each  {x^p).  Thus,  there 


2-45 


exist  unique  smooth  functions  d^,{x,p)  and  u^{x,p),  defined  in  a  neighborhood  of  (0,0),  satisfying 

— (a:,p,d*,M*)  =  0 

^  ^  n 

— =  0 

d,(0,0)  =0 

w*(0,0)=0  (2.121) 

and  such  that 

H{x,p,d,U:,{x,p))  <H*{x,p)  =  H{x,p,d^{x,p),u^{x,p))  <  H{x,p,d^{x,p),u)  (2.122) 

for  each  (x^p,  d,  u)  in  a  neighborhood  of  (x,p,  d,  u)  =  (0, 0, 0, 0). 

ff  we  now  define  V  :7U^  ^  7?.+  to  be  a  (7^,  positive  semidefinite  function  defined  on  a  neighbor¬ 
hood  C/  of  a;  —  0  and  let  Vj  =  take  the  place  of  p  in  (2,122)  and  (2.121),  we  get 

n{x,Vj {x),d,a2{x))  <  (x))))  <  U{x,p,ai[x),u)  (2.123) 

where  we  have  defined 

a:i(a:)  =  d*(a;,  Vj (rr)),  0:2(3;)  =  “*(3;,  Vj (x))  (2.124) 

Thus,  if  we  satisfy  the  so-called  Hamilton- Jacobi  Inequality 

n.{x,Vj{x))<0  (2.125) 

for  all  a;  in  a  neighborhood  of  0  and  set  u  =  0:2(3;)  in  (2.118),  we  obtain  a  closed  loop  system  such 
that 

V.{x)f{x,d,a2{x))-^\\h{x,a2{x)^^^^  (2.126) 

In  solving  (2.125)  we  have  thus  obtained  a  closed  loop  system  with  the  required  (local)  dissipativity 
property,  and  as  shown  in  Section  2.4.2,  we  have  therefore  solved  the  local  nonlinear  state  feedback 
suboptimal  Hoo  control  problem.  Closed  loop  stability  may  be  guaranteed  by  imposing  any  of  the 
conditions  listed  at  the  end  of  Section  2.4.2:  zero-state  detectability  of  (2,118)  or  V  positive  definite 
and  either  solution  of  (2.125)  with  strict  inequality  or  zero-state  observability  of  (2.118). 
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Analagous  to  Section  2.4.2,  we  now  present  a  result  due  to  van  der  Schaft  [66]  relating  the 
solvability  of  the  state  feedback  local  nonlinear  suboptimal  Hqo  control  problem  for  input-affine 
systems 


X  =  a{x)  -{-  b{x)u  -h  9{x)d,  a(0)  —  0 


y  =  x 


z  — 


h{x) 


,  h{0)  -  0 


(2.127) 


to  that  of  its  linearization  at  zero 


X  —  Ax  +  Bvj  -|-  Gd 


y  =  x 


z  = 


Hx 


u 


(2.128) 


where  A  -  (0),  £  -  6(0),  G  -  g(0),  and  H  =  §(0). 

Theorem  2.4*1  Consider  (2,127)  and  (2,128)  and  assume  {H,A)  detectable.  The  following  state¬ 
ments  are  equivalent, 

i,  3  a  linear  state  feedback 


u  =  Kx 


(2.129) 


such  that  the  closed  loop  system  (2,128),  (2,129)  is  asymptotically  stable  and  has  induced  L2 
gain  <  7  (the  linear  state  feedback  Hoo  suboptimal  control  problem  is  solvable), 

a,  3  a  solution  >0  to  the  Riccati  equation 


A'^X  +  XA  +  Xi^GG^  -  BB'^)X  +  =  0  (2.130) 


Hi, 


such  that  all  the  eigenvalues  of  the  matrix  A  —  BB^X  -h  X  have  negative  real  parts, 

3  a  neighborhood  W  of  0,  and  a  nonlinear  state  feedback 

u=^a{x)  (2.131) 
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defined  on  W  such  that  A-\-BK ,  with  K  =  (0)^  is  asymptotically  stable  and  the  closed  loop 

system  (2,127)^  (2.131)  has  locally  induced  L2  gain  <  7  (the  nonlinear  local  state  feedback  Hqo 
suboptimal  control  problem  is  solvable  on  W ). 

Proof:  See  [66].  ■ 

We  note  that  if  we  replace  the  inequality  with  strict  equality  in  (2.125),  we  obtain  the  so-called 
Hamilton- Jacobi-Issacs  equation  [67].  In  solving  the  Hamilton-Jacobi-Issacs  equation,  we  gain  the 
added  familiar  interpretations  for  and  d^  as  the  minimizing  control  and  worst-case  disturbance, 
respectively.  Also,  in  Theorem  2.4.1  above,  technically  we  are  speaking  of  solutions  to  the  Hamilton- 
Jacobi-Issacs  equation.  A  solution  X  to  the  Riccati  equation  in  part  ii  implies  the  existence  of  a 
local  solution  V  to  the  the  Hamilton-Jacobi-Issacs  equation  for  (2.127).  Furthermore,  it  is  true  that 

d^V 

X  =  -^(0)  (2.132) 

and  that  the  linear  part  of  (2.131)  precisely  equals  (2.129)  in  part  i.  Thus,  (2.129)  solves  the 
nonlinear  state  feedback  suboptimal  Hoo  control  problem  on  some  neighborhood  of  the  origin.  It  is 
conjectured  [67]  that  the  domain  of  validity  for  (2.129)  will  always  be  smaller  than  for  (2.131),  but 
it  has  yet  to  be  proven.  , 

We  conclude  this  section  by  making  a  connection  between  Hamilton-Jacobi-Issacs  equations  of 
nonlinear  Hco  control  and  Hamilton-Jacobi-Bellman  equations  for  nonlinear  optimal  control.  To  do 
so,  we  observe  that  if  we  let  7  — >  00,  the  Hamilton-Jacobi-Issacs  equation  for  (2.127)  tends  to  the 
Hamilton-Jacobi-Bellman  equation  of  optimal  control  for  the  system 

X  =  a{x)  -h  b{x)u  (2.133) 

with  the  cost  functional 

rco  1  1 

Jo  (2.134) 

i.e., 

V,(x)a{x)  -  Jv^{x)b{x)b'^{x)Vjix)  +  ^h^{x)hix)  =  0  (2.135) 


2-48 


Local  solutions  for  (2.135)  always  exist  under  the  assumption  of  stabilizability  of  the  linearized 
system  [47].  This  observation  gives  us  a  link  between  nonlinear  Hoo  control  and  optimal  nonlinear 
quadratic  regulation  which  will  be  discussed  further  in  Section  2.5.1. 

We  conclude  this  section  with  the  statement  that  all  the  results  presented  in  this  section  have 
dealt  with  the  suboptimal  Hoo  control  problem.  This  is  the  case  because  complete  theory  for  the 
optimal  case  does  not  yet  exist.  Recently,  however,  approximate  techniques  for  computing  the 
optimal  induced  L2  gain  have  appeared  in  the  literature  [35,  75]. 


2.5  State- Dependent  Riccati  Equation  Techniques 

This  section  presents  the  state-dependent  Riccati  equation  (SDRE)  theory  of  nonlinear  regulation 
for  input-affine  systems  as  recently  developed  by  Cloutier,  D’Souza,  and  Mracek  [13],  and  a  nonlinear 
Hoo  solution  technique  based  on  similar  ideas.  The  section  on  nonlinear  regulation  is  taken  with 
minor  modification  from  [13],  while  the  section  on  nonlinear  Hoo  control  presents  a  solution  technique 
proposed  in  [13]  combined  with  original  work  by  this  author.  It  is  emphasized  that  much  of  the 
theory  in  Section  2.5.1  is  recent  and  unverified.  The  main  contribution  of  this  dissertation  is  to 
verify  the  validity  of  the  technique,  which  is  accomplished  in  later  sections. 


2.5.1  The  Nonlinear  Regulator  Problem 

In  the  nonlinear  regulator  problem,  we  are  interested  in  minimizing  the  infinite-horizon  cost  function 

1 

minimize  J  =  -  [x'^Q{x)x  +  u^R{x)u]  dt  (2.136) 

2  Jto 


subject  to  the  nonlinear  differential  constraint 


X  —  a(x)  -(-  b{x)u 


(2,137) 


given  state  x  E  and  control  u  E  with  a,  5,  i?,  Q  E  >  1,  and  where  Q{x)  = 

H'^{x)H{x)  >  0,  and  R{x)  >  0  for  all  x.  It  is  assumed  that  a(0)  —  0  so  that  the  origin  is  an  open 
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loop  equilibrium  point  of  the  system.  We  seek  stabilizing  solutions  of  the  form 

^  -  L{x)x  (2.138) 

where  the  nonlinear  feedback  gain  L  is  a  matrix  function  of  x.  The  above  formulation  should  be 
familiar  from  linear  quadratic  regulator  (LQR)  theory  [dl,  76]  except  that  the  matrices  Q,  R,  and 
L  all  have  elements  that  are  allowed  to  be  functions  of  x.  Thus,  the  cost  function  (2.136)  may  or 
may  not  be  quadratic  depending  on  whether  Q  and  R  are  constant  matrices  or  not. 

The  SDRE  method  hinges  on  being  able  to  write  the  constraint  dynamics  (2.137)  in  a  pointwise 
linear  structure  having  state-dependent  coefficient  (SDC)  form 

X  ■=  A{x)x  +  B{x)u  (2.139) 

so  that 

a(a;)  =  A{x)x  (2.140) 

and 

b{x)  =  B{x) 

At  this  point  it  is  important  to  note  that  the  assumptions  on  the  open  loop  drift  function  a  guarantee 
that  a  global  SDC  parametrization  of  a  exists  [69].  One  such  factorization  is  [69]: 

A{x)=  [  Va:a{Xx)dX  (2,141) 

Jo 

which  is  guaranteed  to  exist  if  a(0)  =  0  and  a  is  C^. 

We  now  make  the  following  definitions  associated  with  the  SDC  form  for  later  use. 

•  {if  (a;),  A(2;)}  is  a  globally  observable  parametrization  of  the  nonlinear  system  if  the  pair 
{ir(r),  A(r)}  is  observable  for  all  x. 

•  {A(x)^B(x)}  is  a  globally  controllable  parametrization  of  the  nonlinear  system  if  the  pair 
{A(a;),^(r)}  is  controllable  for  all  x. 

•  {ir(r),  A(r)}  is  a  globally  detectable  parametrization  of  the  nonlinear  system  if  the  pair 
{H A{x)}  is  detectable  for  all  x. 


2-50 


•  {A(ir),5(x)}  is  a  globally  stabilizable  parametrization  of  the  nonlinear  system  if  the  pair 
{A{x),B{x)}  is  stabilizable  for  all  x. 

We  associate  with  the  nonlinear  cost  function  (2.136)  the  state-dependent  (algebraic)  Riccati 
equation  (SDRE): 

A^{x)P{x)  +  P{x)A{x)  —  P{x)B{x)R~^ {x)B^ {x)P{x)  H-  Q{x)  =  0  (2.142) 

The  SDRE  nonlinear  regulation  technique  is  to  solve  (2.142),  accepting  only  P{x)  =  P'^{x)  > 
0  V  a;,  and  construct  the  nonlinear  feedback  control  by  setting 

u  =  {x)B'^{x)P{x)x  (2.143) 

so  that  the  nonlinear  feedback  gain  is 

L{x)  =  ~R-^{x)B'^{x)P{x)  (2.144) 

These  equations  can  be  solved  analytically  to  produce  an  equation  for  each  element  of  u,  or  solved 
numerically  at  a  sufficiently  high  sampling  rate,  as  was  done  in  [53].  Note  that  quite  clearly  from 
(2.143),  we  must  have  full  state  feedback  available  in  order  to  construct  the  control  u  using  this 
method. 

The  local  stability  of  the  closed  loop  system  resulting  from  using  the  SDRE  nonlinear  regulation 
technique  is  given  by  the  following  theorem  from  [13]. 

Theorem  2,5,1  In  addition  to  a,  &,  i?,  Q  E  (7^,  k  >  1,  assume  and  A  gives  a  globally  sta~ 

hilizable  and  detectable  state- dependent  coefficient  parametrization  of  the  nonlinear  system.  Then  the 
SDRE  nonlinear  regulation  control  method  has  a  closed  loop  solution  which  is  locally  asymptotically 
stable. 

Proof:  The  proof  is  repeated  from  [13] ,  with  only  minor  notational  changes  to  be  consistent  with 
the  remainder  of  this  dissertation.  Note  that  the  closed  loop  solution  is  given  by 

X  =  [A{x)  —  B(x)R~^(x)B^  {x)P{x)]x  =  F{x)x 
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and  that  the  closed  loop  matrix  function  F{x)  is  guaranteed  to  be  stable  at  every  point  x  from 
standard  Riccati  equation  theory.  Under  the  and  stabilizable/detectable  assumptions  on  the 
system  parameters,  P  e  so  that  F  e  C^.  Expanding  F  in  a  partial  Taylor  series  expansion 
about  zero  yields,  for  some  neighborhood  of  the  origin, 

X  ^  F(0)x  +  i^ix)  •  ||a:|| 


where  'ip{x)  is  of  order  k  and 

lim  'ip(x)  =0 
||x|Ho 

In  a  smaller  neighborhood  about  the  origin,  the  linear  term,  which  has  a  constant,  stable  coefficient 
matrix,  dominates  the  higher-order  term  yielding  local  asymptotic  stability.  ■ 

At  this  point  we  remark  that  the  proof  of  Theorem  2.5.1  relies  on  linearization  arguments,  and 
thus  global  stabilizability  and  detectability  of  the  SDC  parametrization  are  not  required  for  local 
stability,  but  only  stabilizability  and  detectability  of  the  linearization  of  (2.137). 

We  now  consider  necessary  conditions  for  optimality  of  the  SDRE  method.  Prom  the  performance 
index  and  constrained  dynamics  we  form  the  Hamiltonian  function 

H  =  ^x^Q{x)x  +  ^u^R{x)u  +  X^[A{x)x  -f  B{x)u]  (2.145) 

^  A 

with  stationarity  conditions 


o 

II 

(2.146) 

(2.147) 

X  ~  A{x)x  +  B{x)u 

(2.148) 

Using  (2.143)  and  (2.145)  we  have 

(2.149) 

=  R{x)[-R-^{x)B'^{x)P{x)x]  +  B'^{x)\ 

(2.150) 

—  B^{x)\\  —  P(a;)a:] 

(2.151) 
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Thus,  =  0  if  we  choose 


A  -  P{x)x  (2.152) 

Satisfying  Equation  (2.152)  for  all  time  will  satisfy  the  Hu  optimality  condition.  From  here  we  will 
drop  the  argument  (x)  notation  for  simplicity.  Differentiating  (2.152)  with  respect  to  time  gives 

X  =  Pxi-Px  (2.153) 

Using  the  optimality  condition  (2.147)  we  also  have 

A  =  -Qx  -  ^x^QxX  -  ^u^RxU  -  {x'^A^  H-  d-  u^BJ)X  (2.154) 

Equating  (2.153)  and  (2.154)  with  substitutions  from  (2.137)  and  (2.143)  gives 

Px-{-P{Ax-BR~^B'^Px)  =  -Qx-lx’^Q^x-lu^R^u-{x'^Al+A'^+x'^PBR-^B'^)Px  (2.155) 

Rearrange  to  form 

Px  +  Ix'^Q^x  +  lu'^Ra^u  +  x'^AlPx  -  x'^PBR-^Bl Px  +  [A^P  +  PA-  PBR-^B’^P  +  Q]x  =  0 

Jd  £i 

(2.156) 

Flirthermore,  from  (2.142)  note  that  the  term  in  brackets  is  the  nonlinear  regulation  SDRE,  which 
equals  zero,  and  substituting  for  u  one  more  time,  (2.156)  reduces  to 

Px  +  \x'^Q:,x  +  \x^PBR-'^R^R-'^B'^Px  +  x'^AIPx  -  x'^PBR-^B^Px  =  0  (2.157) 

This  is  the  SDRE  Necessary  Condition  for  Optimality  which  must  be  satisfied  for  the  closed  loop 
solution  to  be  locally  optimal.  The  authors  of  [13]  note  that  the  left  hand  side  of  (2.157)  collapses 
to  zero  in  the  standard  case  of  (infinite  time)  linear  quadratic  regulation,  and  thus  this  method  is  a 
true  generalization  of  LQR  theory. 

The  authors  of  [13]  give  the  following  theorem  for  the  scalar  case. 

Theorem  2.5.2  Consider  (2.136)  and  (2.137)  for  scalar  x,  i.e.,  n  =  1.  Then  there  exists  a  unique 
SDC parametrization  for  X  /  0.'  A{x)  —  a{x)/x,  and  the  SDRE  Necessary  Condition  for  Optimality 
is  always  satisfied. 
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Proof:  The  proof  of  uniqueness  is  trivial  but  repeated  here  from  [13]  for  convenience.  Let 

^1(2;)  and  ^2(2;)  be  two  SDC  parametrizations  of  (2.137).  Then  a{x)  —  A\{x)x  =  A2{x)x  so  that 
[Ai(2:)  -  A2{x)]x  =  0.  Thus  ^1(2:)  ~  A2{x)  for  all  x  ^  0.  The  proof  of  satisfaction  of  the  SDRE 
Necessary  Condition  for  Optimality  is  achieved  by  a  straightforward  but  tedious  algebraic  exercise, 
and  is  not  repeated  here.  ■ 

We  remark  that  the  uniqueness  of  A{x)  in  the  scalar  case  could  be  extended  to  a;  =  0  by  assuming 
the  continuity  of  A, 

In  the  multistate  case,  it  is  stated  in  [13]  that  an  infinite  number  of  SDC  parametrizations  exists 
for  (2.137).  The  proof  relies  on  showing  that  at  least  two  parametrizations  exist,  and  that  any  linear 
combination  of  these  two  parametrizations  is  also  a  valid  SDC  parametrization.  The  existence  of  at 
least  two  parametrizations  is  trivial  to  show.  The  existence  of  an  infinite  number  of  parametrizations 
is  more  interesting  to  examine,  and  we  therefore  reproduce  that  part  of  the  proof  here.  Thus,  if  we 
let  ^1(2;)  and  ^2(2;)  be  two  distinct  SDC  parametrizations,  then  a{x)  =  Ai{x)x  =  A2{x)x,  If  we 
consider  the  hyperplane  of  SDC  matrices  ^(2;,  a)  =  aAi[x)  -h  (1  —  a.)A2{x)^  we  find 

A{x^  a)x  =  aAi  {x)x  -h  (1  —  a').A2(2;)2; 

—  aa{x)  H-  (1  “  a)a{x) 

=  aa{x)  H-  a{x)  —  aa{x) 

=  a{x)  (2.158) 

and  thus,  A{Xya)  is  a  valid  SDC  parametrization  for  all  a  E  77,  so  that  there  exists  an  infinite 
number  of  parametrizations  corresponding  to  the  choice  of  a.  The  authors  of  [13]  then  conjecture 
the  existence  of  an  optimal  SDC  parametrization  leading  to  an  optimal  P{x^a)  solution  of  (2.142) 
and  (2.157),  and  propose  a  procedure  for  finding  it  that  results  in  a  two-point  boundary  value  problem 
to  be  solved.  This  author  has  observed  that  the  hyperplane  A(x,  a)  above  does  not  always  span  the 
space  of  valid  SDC  parametrizations,  as  it  considers  only  two  factorizations  as  a  basis,  and  in  the 
two-dimensional  case,  there  may  exist  many  more  valid  parametrizations  which  are  not  obtainable 
as  linear  combinations  of  a  given  pair  of  factorizations.  Thus,  this  procedure  and  argument  requires 
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closer  examination.  From  [30]  however,  we  are  at  least  able  to  verify  the  conjectured  existence  of  an 
optimal  factorization,  but  only  under  certain  assumed  conditions. 

We  conclude  this  section  by  commenting  that,  although  there  are  many  theoretical  issues  to 
be  resolved  in  the  SDRE  nonlinear  regulator  method,  several  impressive  examples  of  solved  design 
problems  are  presented  in  [14].  These  examples  show  excellent  agreement  with  results  obtained  using 
other  well-established  control  techniques,  thus  suggesting  utility  in  the  SDRE  nonlinear  regulation 
method. 


2.5.2  Nonlinear  Control  Via  the  State  Feedback  SDRE  Method 

In  this  section,  we  present  proposed  solution  methods  for  the  nonlinear  Hoo  suboptimal  control 
problem  based  on  the  SDRE  technique.  Solution  approaches  for  input-affine  systems  under  both 
state  and  output  feedback  are  proposed  in  [13],  without  any  proofs  that  the  methods  do  indeed 
work.  We  present  two  original  proofs  for  state  feedback  solution  approaches  to  the  nonlinear  Hoo 
suboptimal  control  problem  based  on  the  SDRE  technique. 

Consider  the  input-affine  nonlinear  system 

X  ~  a{x)  -h  h{x)u  g{x)d 
y  =  x 

z  =  h{x)  +  di2{x)u 

dj^{x)[h{x)di2(x)]  =  [01]  (2.159) 

where  x  €  7^”',  u  €  d  e  ai{x)  e  C^{1  <  i  <  n),  bij{x)  E  (7^(1  <  i  <  n,l  <  j  < 
m),  gij{x)  £  C^{1  <  i  <  n^l  <  j  <  A;  >  1,  for  which  we  desire  to  solve  the  local  nonlinear 
Hoo  suboptimal  control  problem  (see  Section  2.4.1).  Recall  that  2;  in  (2.159)  above  is  our  penalized 
output,  and  the  assumption  in  the  last  line  of  (2.159)  requires  separate  penalties  on  states  and 
controls  and  also  control  penalty  scaling  in  order  to  simplify  the  problem.  Note  that  di2  has  replaced 
R  as  the  control  penalty  matrix  to  reflect  this  scaling,  and  also  to  be  consistent  with  the  notation 
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used  in  the  standard  linear  time-invariant  developments  [18].  In  [13]  the  following  SDRE-based 
solution  approach  to  this  problem  is  presented. 

SDRE  Approach  for  Full  State  Feedback 

i.  Parametrize  (2.159)  in  SDC  form 

X  =  A(x)x  -j-  B(x)u  -h  G(x)d 
y-x 

z  -  H{x)x  +  Di2{x)u  (2.160) 


ii.  Solve  the  Eoo  SDRE 

A^{7:)P[x)-\-P{x)A{x)-P{x)[B{x)B'^{x)  -  ^G{x)G'^ {x)\P{x)  +  H'^{x)H{x)  =  0  (2.161) 

with  7  sufficiently  large  so  that  the  stability  and  complementarity  properties  (see  Section  11.1) 
hold  in  order  to  obtain  P{x)  >  0  V  a;. 

iii.  Construct  the  nonlinear  Hoo  feedback  control  via 

u{x)  =  -B'^{x)P{x)x  (2.162) 

In  order  to  prove  the  validity  of  an  SDRE-based  method,  we  need  to  strengthen  the  assumptions 
on  (2.159)  to  agree  with  those  of  Section  2.4.  This  we  now  do  as  we  formally  state  and  prove  our 
result. 

Theorem  2.5.3  Consider  (2A59)  and  assume  z  E  h{0)  —  0.  Also  assume  that  all  mappings  in 
(2.159)  are  and  that  {H{0),A{0)}  is  detectable  and  {^(0),P(0)}  is  stahilizable.  Then  the  state 
feedback  SDRE  design  procedure  given  by  (2.161)  and  (2.162)  yields  a  local  solution  to  the  nonlinear 
Hoo  control  problem  for  (2.159). 

Proof:  We  consider  the  Hamiltonian  point  of  view  of  solving  the  local  nonlinear  Hqo  suboptimal 

control  problem  for  a  fixed  7,  as  in  Section  2.4.3.  First,  we  note  that  (2.159)  satisfies  Assumption 


2-56 


2.4.1  in  that  all  controls  have  an  independent  linear  penalty  placed  on  them.  Thus,  the  theory 
of  Section  2.4.3  may  be  applied  to  this  problem,  and  our  goal  is  to  seek  a  (locally)  positive 
semidefinite  solution  to  the  HJI  (2.125).  Recalling  (2.119)  we  define  the  Hamiltonian 

n{x,Vx,d,u)  =  Vx{x)f{x,d,u)  +  \\z{x,u)\\^  -  7^||d|P  (2.163) 


Using  (2.159),  (2.163)  becomes 

H{x,Vx,d,u)  -  Vx{x)[a{x)  +b{x)u  g{x)<]\  +  h^(x)h{x)  +u^u  -j^d^d  (2.164) 

We  propose  a  solution  of  the  form  V{x)  —  x'^P(x)x,  with  P{x)  =  P'^(x).  Using  the  proposed  form 
for  V,  we  have  that 

Vx(x)  =  2x^P{x)  +  x'^[—x\  (2.165) 

If  we  define  the  state-dependent  matrix 

Mp{x)  =  l^x]  (2.166) 

and  use  the  state-dependent  parametrization  (2.160),  (2.164)  can  be  written  as: 

H{x,  Vx,d,  u)  =  x^{2P  4-  Mp)[Ax  +  Bu-\-  Gd\  -h  x^H^Hx  -t-  vFu  -  'fd^d  (2.167) 


where  we  have  dropped  the  (x)  dependency  notation  for  convenience.  Now,  the  first-order  necessary 
conditions  (see  (2.121)) 


give 


=  0  =  x^(2P  -+  Mp)G  -  2'fdl 
=  0  =  a:'^(2P  -t-  Mp)B  +  2uJ 


(2.168) 

(2.169) 


d.  =  ^G'^(2P  +  Mpfx  (2.170) 

=  -iB^(2P  +  Mpfx  (2.171) 


Thus,  the  HJI  we  want  to  solve  becomes 


x'^{2P  -+  Mp)[A  -  ipP^(2P  +  Mpf  +  ^GG'^{2P  +  Mpf]x  +  x'^H'^Hx 
+^x'^{2P  +  Mp)BB'^{2P  +-  Mpfx  -  ^x'^{2P  ■+-  Mp)GG'^[2P  -I-  Mpfx  <  0  (2.172) 
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Using  simple  algebra  and  the  fact  that  Zx  —  (2.172)  simplifies  to 


x'^[A^P  +PA-  P(BB'^  -  ^GG'^)P  +  H'^H]x  +  x'^[MpA 

-^Mp{BB'^  -  ^GG^)M'f  -  Mp{BB'^  -  ^GG‘^)P\x  <  0  (2.173) 

We  observe  that  if  we  set  the  first  bracketed  term  in  (2.173)  equal  to  zero,  we  obtain  the  Hoo  SURE 
(2.161),  and  the  remaining  inequality 

N{x)  =  x'^[MpA  -  -^Mp(BB'^  -  -  Mp{BB'^  -  ^G(f')P]x  <  0  (2.174) 


Now,  since  a  and  h  were  assumed  to  be  (7^,  they  can  be  expanded,  at  least  locally,  in  a  Taylor 
series  about  the  origin  as 


a{x)  = 

-  “(0)  +  1 

fda\ 

1  'p  j 

(fc) 

x  +  -x^  \ 
1=0  ^ 

h{x)  - 

=  h(fi)  “h 

(i) 

1  Tl 
X  +  -X  \ 

x=0  ^ 

(d'^h\ 

UVx=o''^"' 

(2.175) 

and  we  recall  that  by  assumption  a(0)  =  0,  h{0)  =  0.  Thus,  recalling  the  SDC  parametrization 
(2.160)  we  can  write 


A{x)x  = 
H{x)x  — 


1 


T  ( 


da\ 

'  2"  \dx^J^^Q 
m\  1  T  (^\ 


+  ■■■ 

+  ■■■ 


(2.176) 


Thus,  A{x)  and  H{x)  are  made  up  of  polynomials,  so  that  (2.161)  implies  that  P  will  be  made  up 
of  polynomial  functions  of  x.  The  definition  of  polynomial  equality  requires  all  coefficients  on  like 
powers  of  x  to  be  equal,  so  that  from  this  discussion  it  is  clear  that,  if  P{x)  solves  (2.161),  the 
constant  coefficients  must  match  and  we  must  have 


A^{0)P{0)  +  P(0)^(0)  -  P(0)[B(0)B^(0)  -  ^G(0)G^(0)]P(0)  +  H^{0)H{0)  =  0  (2.177) 

Recalling  the  result  of  van  der  Schaft  [66]  at  the  end  of  Section  2.4.2,  the  solution  P{x)  of  (2.161)  has 
a  Oth-order  (constant)  term  P(0)  >  0  which,  by  assumption,  is  also  a  stabilizing  solution  of  (2.177). 
Therefore,  it  must  necessarily  solve  the  local  Hoo  suboptiraal  control  problem  for  the  linearized 
system  about  the  origin.  Furthermore,  recall  that  any  solution  of  the  linearized  problem  is  at  least 
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a  local  solution  of  the  nonlinear  Hoo  suboptimal  control  problem.  Thus,  the  control  (2.171)  locally 
solves  the  nonlinear  Hoo  suboptimal  control  problem.  A  point  to  be  made  is  that  we  have  not 
addressed  the  inequality  N  <0.  This  is  because  every  term  in  this  expression  has  at  least  one  factor 
of  the  matrix  function  Mp  in  it.  Recalling  (2.166),  we  see  that  Mp{0)  =  0,  so  that  the  Oth-order 
contribution  of  iV  to  (2.173)  is  zero.  Thus,  by  satisfying  (2.161)  we  satisfy  (2.177),  and  the  HJI 
(2.173)  is  indeed  satisfied  to  second  order  in  x  and  thus  is  satisfied  in  some  neighborhood  of  the 
origin,  verifying  that  P{x)  and  give  a  local  solution  to  the  nonlinear  Hqq  suboptimal  control 
problem,  so  that  the  theorem  is  proven.  ■ 

Note  that  positive  semidefinite  P(0)  is  not  as  strong  a  requirement  as  in  the  state  feedback  approach 
of  [13],  which  requires  a  positive  definite  solution  P{x)  V  x,  which  is  of  course  sufficient  to  guarantee 
P(0)  >  0.  We  also  note  that,  just  as  in  the  case  of  the  SDRE  nonlinear  regulator,  since  the  above 
arguments  depend  on  linearized  analysis,  the  global  stabilizability  and  detectability  assumptions  on 
the  SDC  par ametrizat ions  have  been  relaxed  to  detectability  of  {J7(0),  A(0)}  and  stabilizability  of 
{A(O),B(0)}. 

Finally,  recalling  Mp(0)  =  0  and  taking  P(0)  from  (2.177),  we  see  that  the  linear  part  of  the 
optimal  control  (2.171)  is  —  -B'^{0)P{0)x.  Thus,  although  (2.162)  does  not  include  the  Mp 
term  seen  in  (2.171),  the  linear  parts  of  both  controls  are  identical,  and  thus  the  full  state  feedback 
approach  of  Cloutier  et  al  [13]  is  valid  if  the  proper  strengthening  of  assumptions  indicated  above 
is  performed. 

An  alternative  approach  to  solving  the  nonlinear  Poo  control  problem  by  the  SDRE  method  may 
be  derived  not  by  proposing  a  form  for  the  Lyapunov  function  V{x)  —  x'^P{x)x,  P{x)  =  P^{x)^ 
but  instead  proposing  a  form  for  the  partial  derivative  (gradient)  of  the  Lyapunov  function,  Vx{x)  ~ 
2x^ P{x)^  with  P(x)  a  symmetric  matrix  function  of  x.  To  see  this,  we  first  determine  the  HJI  we 
wish  to  solve  in  terms  of  Vx{x).  Again  dropping  the  (x)  dependency  for  ease  of  notation,  directly 
from  (2.164)  the  first-order  necessary  conditions  (see  2.121)  give 
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m 

du 


U~U^ 


(2.179) 


=  0  =  I4B  +  2ul 


so  that 


u.  =  -\b'^vJ 


Substituting  the  above  back  into  (2.164)  and  simplifying  gives  the  HJI 


(2.180) 

(2.181) 


Va^Ax  -  +  ^V^GG'^Vj  +  x'^H^Hx  <  0  (2.182) 

We  now  substitute  the  proposed  form  14  =  2x^P  into  (2.182)  and  simplify  (again  using  x^Zx  = 
x'^Z^x)  to  obtain 

x'^[A'^P  +  PA  -  P{BB'^  -  ^GG'^)P  +  H'^H]x  <  0  (2.183) 

7 

and  we  see  comparing  to  (2.173),  that  we  have  eliminated  the  second  bracketed  term,  leaving  only 
the  left  hand  side  of  the  SDRE  proposed  in  (2.161)  that  must  be  set  less  than  or  equal  to  zero 
to  solve  the  nonlinear  iJoo  problem.  While  it  is  significantly  easier  to  solve  the  HJI  (2.183)  as 
compared  to  (2.173),  the  new  wrinkle  in  this  solution  approach  is  that  in  the  multistate  case  we  must 
solve  the  set  of  simultaneous  partial  differential  equations  T4  (3^)  —  2x^P{x)  to  obtain  a  Lyapunov 
function  candidate  V{x),  which  we  must  then  hope  is  at  least  locally  positive  semidefinite  in  order 
to  guarantee  local  asymptotic  stability.  The  conditions  under  which  this  may  be  done  are  explored 
in  Sections  4.4  and  4.5.  We  thus  have  an  apparent  tradeoff  between  these  two  approaches.  In  the 
first  method  we  have  a  Lyapunov  function  V{x)  =  x^P{x)x  which  we  can  guarantee  to  be  locally 
positive  semidefinite  if  P(0)  is  a  positive  semidefinite  matrix,  and  we  have  conditions  (linearized 
stabilizability  and  detectability)  that  tell  us  when  this  will  be  the  case.  However,  we  have  a  diflBicult 
HJI  to  solve,  involving  two  parts,  the  second  of  which  contains  coupled  terms  of  the  matrices  P 
and  Mp.  In  the  second  method,  wo  need  only  solve  the  SDRE  (2.161)  to  get  a  negative  definite 
Vj  but  then  it  is  much  more  difficult  to  deduce  the  existence  of  a  positive  semidefinite  Lyapunov 
function  V,  An  interesting  observation  is  that,  using  this  second  approach,  we  always  obtain  the 
same  optimal  control  as  proposed  by  Cloutier  et  al  [13].  A  final  comment  that  supports  the  second 
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proposed  approach  is  the  fact  that  as  7  oo,  (2.182)  and  (2.183)  (in  the  equality  case)  approach 
the  correct  Hamilton- Jacobi-Bellman  equation  (2.135)  (properly  accounting  for  the  factor  of  1/2 
missing  in  (2.164))  and  nonlinear  regulator  SDRE  (2.161),  as  they  should,  while  (2.173)  does  not. 
This  second  approach  is  thus  consistent  with  linear  Hoo  control  approaches,  which  is  appealing  from 
an  intuitive  perspective.  The  relationship  between  these  two  design  approaches  is  further  explored 
in  Section  4.5. 

In  this  chapter  we  have  reviewed  relevant  theory  for  stability  analysis  and  control  synthesis  for 
nonlinear  dynamic  systems.  Except  for  Section  2.5.2,  in  which  we  gave  an  original  proof  that  the 
SDRE  method  does  provide  a  local  solution  of  the  nonlinear  Hoc  suboptimal  control  problem  under 
suitable  assumptions,  all  the  material  was  compiled  from  existing  references,  to  serve  as  background 
for  the  remainder  of  this  dissertation. 
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III.  Motivational  Example  Problem 


In  this  chapter,  we  apply  the  methods  of  Chapter  2  to  an  academic  second-order  single-input  example 
problem.  This  application  is  intended  to  clarify  the  theory  presented  in  Chapter  2,  to  highlight  the 
differences  between  the  various  nonlinear  control  design  methods,  and  to  point  to  promising  areas  of 
research.  The  problem  to  be  considered  is  taken  from  [34],  and  has  several  interesting  aspects  which 
will  be  pointed  out  as  they  are  uncovered.  In  Section  3.1  we  present  the  problem  and  perform  some 
introductory  analysis.  In  the  remaining  sections  of  the  chapter  we  apply  each  of  the  four  control 
methods  of  Chapter  2  in  the  same  order  they  were  presented  there. 


3,1  Problem  Setup  and  Introductory  Analysis 


Consider  the  system 


_ 

_ 

“ 

Xi 

X1X2 

j  2;  — 

X2  -  x\ 

^2 

x\  -j~  d  -f  'u 

u 

for  which  we  immediately  identify  from  our  standard  notation 

X  —  a{x)  +  b{x)u  +  g{x)d  (3.2) 


that 


X1X2 


(3.3) 


Our  control  objective  is  to  stabilize  (3.1)  while  simultaneously  attenuating  the  effect  of  the  distur¬ 
bance  d  on  the  penalized  output  z,  assuming  full  state  feedback.  Thus,  the  problem  we  are  trying 
to  solve  may  be  most  naturally  cast  as  a  state  feedback  nonlinear  Hoo  control  problem,  although 
the  other  methods  of  Chapter  2  may  also  be  used  at  least  to  attempt  stabilization.  Stability  and 
disturbance  attenuation  in  the  Hoo  sense  was  the  premise  of  [34],  in  which  the  authors  used  this  ex¬ 
ample  problem  for  the  stated  reason  that  (3.1)  cannot  be  stabilized  by  any  linear  full  state  feedback 
control  law  u  =  axi  -\-  bx^.  This  claim,  if  true,  provides  excellent  motivation  for  employing  nonlinear 
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control  techniques,  and  also  implies  that  the  open  loop  system  is  itself  unstable.  We  will  now  use 
the  methods  of  Section  2.1  to  verify  this  important  claim. 

In  stability  analysis  of  (3.1),  we  wish  to  first  consider  the  unforced  system  (i.e.,  with  d  =  0,  -w  =  0) 
without  penalty  variable: 

Xi 
^2 

Clearly,  (3.4)  has  equilibrium  points  at  xi  —  0  for  all  X2  E  11.  Thus  the  a;2-axis  of  the  xiX2  phase 
plane  consists  entirely  of  critical  points.  We  shall  now  examine  the  stability  of  these  critical  points 
via  the  techniques  of  linearization  and  first  integrals  [68].  The  Jacobian  of  (3.4)  at  any  critical  point 
(0,2^2)  is  easily  computable  as 


(3.4) 


J{0,X2) 


■ 

“ 

X2 

Xi 

X2 

0 

2xi 

0 

(0,a;2) 

0 

0 

(3.5) 


Thus,  each  positive  2;2“axis  critical  point  has  a  1-dimensional  unstable  manifold  and  a  l-dimensional 
center  manifold,  and  each  negative  X2  axis  critical  point  has  a  1-dimensional  stable  manifold  and  a 
1-dimensional  center  manifold.  It  is  quite  trivial  to  calculate  the  directions  of  the  stable/unstable 
manifolds  as  parallel  to  the  a^i-axis,  and  the  direction  of  the  center  manifolds  as  the  X2-axis.  The 
origin  {xi  =0,  X2  =  0)  clearly  has  a  2-dimensional  center  manifold.  The  existence  of  these  center 
manifolds  is  one  of  the  interesting  aspects  of  this  problem  alluded  to  previously.  As  in  [34],  we 
are  most  interested  in  the  behavior  around  the  critical  point  at  the  origin.  It  turns  out  that  (3.4) 
is  separable  and  can  be  solved  explicitly  to  obtain  a  first  integral  of  the  motion,  which  yields  a 
parametrized  set  of  trajectories  which  solutions  of  (3.4)  must  follow  in  the  phase  plane.  This  can  be 
seen  by  dividing  the  ±2  equation  by  the  equation  to  obtain 

(3.6) 

dxi  X1X2  X2 


Rearranging  gives 


X2dX2  —  Xidxi 


(3.7) 
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and  integrating  and  multiplying  by  2  yields 


X2  =  (7  G  7^ 


(3.8) 


Thus,  solutions  of  (3.4)  follow  hyperbolic  trajectories  parametrized  by  the  constant  C  which  can  be 
determined  from  the  initial  conditions.  For  zero  initial  conditions,  (3.8)  reduces  to  the  degenerate 
hyperbola 


or  equivalently  to  the  orthogonal  lines  X2  —  which  are  the  center  manifolds  of  the  critical 

point  at  the  origin.  For  any  nonzero  initial  condition  near  the  origin  (arbitrarily  close)  which  is 
not  a  critical  point,  the  solution  of  (3.4)  follows  a  hyperbolic  trajectory  to  infinity,  and  is  therefore 
unbounded,  clearly  indicating  instability  of  the  origin  for  the  unforced  system. 

We  now  wish  to  consider  how  things  change  allowing  only  the  presence  of  the  control  input  u. 
We  first  consider  the  linearization  of  (3.1),  and  attempt  to  invoke  Theorem  2.2.8.  Through  simple 
calculation  one  obtains  the  linear  approximation  x  =  Ax  Bu  to  (3.1)  as 


" 

- 

■ 

Xi 

— 

0 

0 

X  + 

0 

X2 

0 

0 

1 

(3.9) 


yielding  the  controllability  matrix 


0  0 
1  0 


(3.10) 


which  clearly  has  rank  one.  Thus,  the  linearization  has  one  uncontrollable  mode,  corresponding  to 
a  zero  eigenvalue  in  the  ^4  matrix.  Thus,  Theorem  2.2.8  provides  no  information  as  to  whether  or 
not  (3.1)  can  be  locally  asymptotically  stabilized  by  either  linear  or  nonlinear  feedback,  and  (3.1) 
represents  a  critical  system  for  which  we  need  a  stronger  analysis  technique  than  the  Principle  of 
Stability  in  the  First  Approximation.  We  therefore  attempt  to  verify  the  nonstabilizability  of  (3.1) 
via  linear  feedback  by  the  Center  Manifold  Theorem,  Theorem  2.1.7,  and  its  associated  Reduction 
Principle,  Theorem  2.1.8. 
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Under  an  arbitrary  linear  full  state  feedback,  u  =  axi-\-hx2^  a,  6  G  7?.,  (3.4)  (with  d  =  0)  becomes 

(3.11) 


±1 

X2 

x\  +  axi  -1-  hx2 

which  has  Jacobian  at  the  origin 

J(0,0)  = 


X2  Xy 
2xi  +  a  b 


J  (0,0) 


0  0 
a  b 


(3.12) 


from  which  it  can  be  seen  that,  for  such  a  control  to  stabilize  (3.1),  h  must  be  chosen  negative.  This 
may  not  be  sufficient,  however,  as  the  behavior  along  the  center  manifold  whose  existence  is  implied 
by  the  zero  eigenvalue  of  (3.12)  may  be  unstable.  Following  the  procedure  outlined  in  Section  2.1.2, 
we  can  write  (3.11)  in  the  form  of  (2.21),  (2.22)  by  identifying  the  matrix  of  right  eigenvectors  of 

(3.12)  as 

f  .  ol 

(3.13) 


r  = 


and  performing  the  coordinate  transformation 


b  0 
—a  1 


\ 

y 

Vv 


=  T-^x 


(3.14) 


to  give 


y  =  Ay  +  g(y,  z)  =  by  +  ayz  +  z^(6^  -  a^) 
z  =  Bz-\-  h{y^  z)  =  Oz-^  z{y  -  az) 

We  now  seek  to  solve  the  center  manifold  equation  with  y  =  'k{z) 

dTT(z)^ 


(3.15) 


dz 


-{Bz  +  h(7r(2:),  z))  =  ATr{z)  +  fl(7r(z),  z) 


(3.16) 


or,  making  the  appropriate  associations  from  (3.15) 

dT(z) 


dz 


(z(7r(z)  —  az))  =  bTr{z)  +  azir(z)  +  z^{lr  —  a?) 


(3.17) 


3-4 


We  solve  (3.17)  approximately  by  letting  ^{z)  =  az"^  +  0{z^)  and  gathering  like  terms  in  powers  of 
2  to  yield 


IP 


a  = 


(3.18) 


Thus,  the  behavior  along  the  center  manifold  is  governed  by  the  equation 

-  62 


-22  +  Oiz^ 


~  — 


(3.19) 


Invoking  Lemmas  2.1.2  and  2.1.3,  we  conclude  that  (3.19)  is  unstable  for  all  values  of  a,  and  thus 
by  Theorem  2.1.8  indeed  we  find  that  (3.1)  is  not  asymptotically  stabilizable  by  any  linear  full  state 
feedback  n  ~  axi  -|-  bx2-  Thus,  we  have  well  motivated  the  need  to  seek  nonlinear  feedbacks  to 
stabilize  (3.1),  which  we  now  proceed  to  do. 


3.2  Feedback  Linearization 


In  this  section  we  attempt  to  apply  the  techniques  of  Section  2.2  to  (3.1).  In  particular,  since  (3.1)  is 
a  single- input  system,  we  will  be  drawing  heavily  on  the  material  of  Section  2.2.2.  We  first  consider 
the  State  Space  Exact  Linearization  Problem  for  (3.1),  since  if  we  can  solve  it,  we  can  at  least 
stabilize  the  origin.  Recall  that  in  the  terminology  of  Section  2.2.2  we  have  h{x)  =  [0  1]^  and  n  =  2, 
and  from  Theorem  2.2.3  that  the  problem  is  solvable  in  a  neighborhood  of  the  origin  if  the  following 
two  conditions  hold: 


i.  the  matrix  [b{0)  adab{0)]  has  rank  2 

ii.  the  distribution  D  =  span{b}  is  involutive  near  x  =  0. 
Using  the  definition  of  the  Lie  bracket  (2.36),  we  find 


- 

■ 

“ 

" 

■ 

_ 

“ 

0 

0 

^2 

Xi 

0 

-Xi 

0 

0 

. 

2xi 

0 

1 

0 

(3.20) 


and 


[6,  Cidfj6]  — 


-1  0 

0  0 


0  0 
0  0 


-Xi 

0 


(3.21) 
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From  (3.20)  we  see  that  the  matrix  in  item  i  above  is 


0  -Xi 


0  0 


(3.22) 


1  0 


1  0 


L  J  (0,0)  L  J 

which  clearly  does  not  have  rank  2,  and  thus  the  SISO  State  Space  Exact  Linearization  Problem  is 
technically  not  solvable  for  (3.1)  at  the  origin.  Recalling  (2.43),  we  see  that  the  distribution  D  is 
indeed  involutive,  however,  so  that  item  ii  is  satisfied.  The  failure  of  item  i  above  is  associated  with 
the  lack  of  existence  of  a  suitable  output  function  for  which  (3.1)  has  a  well-defined  relative  degree, 
r  =  2,  at  the  point  (0,  0),  which  in  turn  implies  a  lack  of  controllability'  when  Xi  =  0.  Prom  (3.22) 
we  see  that,  if  we  consider  the  problem  at  any  point  for  which  Xi  ^  0,  we  satisfy  item  i.  Since  every 
point  with  Xi  =  0  is  an  equilibrium,  however,  it  may  not  be  crucial  to  control  the  system  at  those 
points.  Thus,  although  item  i  above  is  not  satisfied,  we  will  proceed  with  the  design  procedure  in 
hopes  of  obtaining  a  stabilizing  control.  We  seek  a  scalar  output  function  y  —  c{x)  such  that 


r 

L,c=-b  =  0 

(3.23) 

(3.24) 

Equation  (3.23)  implies 


dc  dc 
dxi  dx2 


dc 

dxo 


=  0  ^  c  ==  c{xi ) 


so  that  c  is  a  function  of  xi  only.  Equation  (3.24)  implies 


’  dc  dc 
dxi  dx2 


-Xi 

0 


-  I  -/n 


(0,0) 


which  can  be  solved  by  (arbitrarily)  letting 


(3.25) 


(3.26) 


(9c  _  1 

dxi  Xi 


so  that 


c{xi)  —  ln{xi). 


(3.27) 
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Taking  our  new  coordinates  as  in  Lemma  2.2.3,  we  get 

zi  =  c{xi)  =  ln{xi)  (3.28) 

=  0^2  (3.29) 

and  we  see  that  this  transformation  is  well-defined  only  for  Xi  >  0.  Although  we  do  not  have  a 
coordinate  transformation  valid  in  a  neighborhood  of  the  origin,  we  will  proceed  with  the  design 
steps  to  illustrate  the  theory,  and  consider  possible  modification  of  the  control  scheme  to  account 
for  this  problem  later.  Recall  that  the  linearizing  feedback  can  be  found  via 

u  =  (j){x)  -f-  6{x)v  (3.30) 


Z2  -  LaC{Xi)  = 


Xi 


X1X2 


with 

0(^)  = 
9{x)  = 

Computing  the  necessary  values  we  find 

Llc^lO  1] 


~Llc{x) 

LtLaC{x) 

1 

LbLaC{x) 


XIX2 


L^LaC  —  [0  1] 


=  1 


(3.31) 

(3.32) 


(3.33) 


(3.34) 


so  that  the  linearizing  feedback  is 

u  =  ~x\  +  V  (3.35) 

and  we  see  that  this  feedback  control  is  well-defined  for  all  x  despite  the  above-mentioned  restricted 
region  of  applicability  of  the  coordinate  transformation.  In  the  tranformed  set  of  coordinates,  (3.1) 
becomes 

f.l  fo  J  ui 

(3.36) 


- 

- 

■ 

" 

- 

k 

— 

0 

1 

+ 

0 

k 

0 

0 

Z2 

1 
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The  transformed  system  (3.36)  has  clearly  been  rendered  linear  by  the  feedback  (3.35),  and  a  simple 
calculation  gives  a  controllability  matrix  of 


0  1 


(3.37) 


[1  OJ 

so  that  (3.36)  is  clearly  seen  to  be  controllable.  Now,  if  the  coordinate  transformation  were  valid  in 
a  neighborhood  of  the  origin,  we  could,  for  example,  choose  v  —  azi  4-  hz2  with  h  negative  and  large 
enough  and  a  negative  and  small  enough  so  that  the  eigenvalues  of  the  closed  loop  system 


" 

- 

■ 

k 

0 

1 

Zl 

a 

h 

Z2 

(3.38) 


given  by 


h  ±  (6^  +  4a)  2] 


(3.39) 


are  both  negative,  thus  ensuring  stability  of  the  closed  loop  system.  Since  the  coordinate  transforma¬ 
tion  is  not  locally  valid,  however,  we  cannot  guarantee  closed  loop  stability  using  this  control.  When 
the  coordinate  transformation  (3.27)  is  undefined  (when  Xi  <  0),  it  might  be  possible  to  modify  the 
above  control  scheme  to  provide  stability,  but  this  would  probably  require  some  ad  hoc  techniques 
such  as  switching  logic  based  on  the  sign  of  in  the  controller,  and  criteria  to  tell  the  controller  not 
to  try  to  evaluate  zi  when  Xi  —  0.  These  considerations  are  beyond  the  scope  of  feedback  lineariza¬ 
tion  theory,  and  will  therefore  not  be  pursued.  Alternatively,  recall  from  (3.22)  that  the  conditions 
for  solvability  of  the  SISO  State  Space  Exact  Feedback  Linearization  Problem  are  satisfied  at  any 
point  for  which  Xx  7^  0.  Following  the  above  steps,  it  is  easy  to  show  that  if  we  consider  a  point 
{xi^x^)  =  (^,0),  A;  /  0,  by  defining  our  output  function  as  c{x)  =  (3.1)  can  be  transformed 

via  the  valid  (in  a  neighborhood  of  {xi^X2)  —  (A;,  0))  coordinate  transformation  Z2)  =  {xi^xix^) 
to  (3.36)  above.  Thus,  we  could  create  and  locally  stabilize  a  critical  point  somewhere  close  to  the 
origin,  but  our  neighborhood  of  attraction  would  never  include  the  a;2-axis.  In  summary,  from  the 
above  discussion  we  have  seen  that  not  meeting  the  technical  requirements  for  exact  state  space 
feedback  linearization  of  (3.1)  efiPectively  prevents  us  from  using  feedback  linearization  theory  to 
provide  a  control  scheme  for  stabilizing  the  origin  of  (3.1)  as  desired. 
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Accepting  this,  for  tutorial  purposes  we  nevertheless  suppose  that  we  have  obtained  a  valid 
coordinate  transformation,  and  proceed  with  the  disturbance  attenuation  part  of  the  problem.  In 
the  transformed  state  space  we  find  that  our  system  becomes 


- 

- 

“ 

“ 

- 

- 

Wi 

0 

1 

Wi 

+ 

0 

d  + 

0 

W2 

0 

0 

W2 

1 

1 

(3.40) 


W2  ” 


(3.41) 


1^  -h  i;  J 

and  we  see  that,  even  though  we  have  assumed  a  solution  to  the  SISO  State  Space  Exact  Linearization 
Problem,  the  input-output  behavior  of  our  system  remains  highly  nonlinear.  Thus,  we  cannot  employ 
linear  control  theory  directly  to  solve  the  problem.  We  could,  however,  employ  linear  iJoo  control 
theory  on  the  linearization  of  (3.41)  (with  (3.40)  unchanged  since  it  is  already  linear)  to  obtain  a 
local  solution  to  the  problem,  as  per  Theorem  2,4.1.  To  do  so  we  must  solve  the  state  feedback 
algebraic  Riccati  equation 


A^X  +  XA  +  Xi^GG'^  -  BB'^)X  +  =  0 


(3.42) 


where  from  the  linearization  of  (3.41)  we  have 


H  = 


Letting 


X  = 


-2 

1 

-2 

0 

Xu 

Xl2 

Xl2 

^22 

and  substituting  the  appropriate  values,  (3.42)  becomes 


1 

O 

o 

_ 1 

Xii  Xi2 

1 - 

1 _ 

0  1 

1  0 

^12  X22 

Xi2  X22 

- 1 

O 

o 

_ 1 

- 

■ 

■* 

■ 

“ 

- 

xn 

Xl2 

0 

0 

Xii 

Xi2 

-2  -2 

H-A 

Xl2 

X22 

0 

1 

Xi2 

X22 

-h 

1  0 

-2  1 

-2  0 


0  0 
0  0 


(3.43) 
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where,  as  in  [34],  we  have  defined 


A-  — -1 


For  illustrative  purposes,  we  shall  solve  (3.42)  for  two  values  of  A.  It  can  be  seen  from  (3.43)  that 
we  need  7  >  1  to  be  guaranteed  existence  of  a  stabilizing  solution.  We  shall  therefore  consider  the 
suboptimal  Hoo  control  problem  for  7^  =  ^  {alter natively,  2)  so  that  we  have  correspondingly 


A  —  -  {alternatively,-^).  Note  that  7^ 


100 

99 


is  closer  to  the  optimal  value  of  one  than  is 


7^  =  2.  Simplifying  (3.43)  results  in  three  nonlinear  equations  to  solve  simultaneously: 


^^12  H"  S  —  0 


2^12  -f  Xxl2  -h  1  =  0 


xii  +  XX12X22  —  2=0 


(3.44) 


For  A  =  —  |,  (3.44)  can  be  solved  to  give  the  stabilizing  solution  of  (3.43) 


a:  = 


2  +  6V5  4 

4  Sy/2 


which  yields  the  control 


V  =  —B^Xw  =  —iwi  —  3\/2w2 


Thus,  the  control  for  the  original  system  for  A  =  —^  is  given  by 

u  =  —xl  —  4ln{xi)  —  3\/2x2 


(3.45) 


(3.46) 


(3.47) 


For  A  =  “1^5  we  obtain  the  stabilizing  solution  of  (3.43) 


3.2  20v^ 

20^/2  75.87 


which  yields  the  control 


V  =  —B^Xw  =  —20V2wi  —  75-9w;2 


so  that  u  for  the  original  system  for  A  =  -  is  given  by 


(3.48) 


(3.49) 


.  =  —Xi  —  20y^ln{xi)  -  75.9x2 


(3.50) 
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EKamination  of  (3.47)  and  (3.50)  shows  that  neither  control  is  defined  for  xi  <  0,  as  expected  from 
our  previous  discussion,  and  also  that  the  near-optimal  solution  (for  A  =  —  requires  significantly 
greater  control  effort  as  we  might  expect,  since  we  are  attempting  a  greater  level  of  disturbance 
attenuation. 

Thus,  assuming  we  had  a  valid  coordinate  transformation,  state  space  exact  feedback  linearization 
would  allow  us  to  solve  the  Hqo  suboptimal  control  problem  at  least  locally.  This  result  is  significant 
because,  if  we  had  tried  to  solve  the  Hoo  suboptimal  control  problem  for  the  original  system  via  its 
linearization,  we  would  obtain  u  =  —V^X2,  and  thus  would  have  failed  to  stabilize  the  system  since 
we  have  already  shown  that  (3.1)  cannot  be  stabilized  by  any  linear  feedback.  We  will  say  more 
about  the  solution  given  above  later,  in  comparing  it  to  solutions  obtained  by  other  design  methods. 

Although  the  solution  above  would  be  a  valid  local  solution  to  the  nonlinear  Hqo  suboptimal 
control  problem,  ideally  we  would  like  to  solve  this  problem  by  completely  feedback  linearizing  the 
nonlinear  input-output  behavior,  and  then  using  linear  Hoo  theory  on  the  resulting  system  to  obtain 
a  larger  region  of  validity  for  the  control  than  what  we  get  by  using  the  linearization  of  the  output 
equation.  Thus,  we  now  seek  a  solution  to  the  input-output  feedback  linearization  problem  for  (3.1). 

Recall  from  Section  2.2.2  that  a  sufficient  condition  for  a  solution  to  exist  to  our  input-output 
linearization  problem  for  a  SISO  system  with  specified  output  is  that  the  system  have  relative  degree 
r  =  7iioT  the  specified  output.  This  is  equivalent  to  the  specified  output  solving  the  state  space  exact 
linearization  problem,  which  z  =  X2  —  xf  clearly  does  not,  since  it  does  not  solve  (3.26).  We  know 
this  is  sufficient  because  of  Theorem  2.4.1.  From  Section  2.2  it  is  clear  that,  if  for  the  given  z  we  have 
some  well-defined  relative  degree  r,  we  can  obtain  a  linearized  input-output  relationship.  Further, 
if  the  zero  dynamics  of  the  transformed  system  are  LAS,  we  could  use  any  stabilizing  control  on 
the  linearized  subsystem  (including  one  from  linear  Hoo  theory)  to  guarantee  stability  of  the  overall 
closed  loop  system.  It  is  also  true  that  if  we  did  use  the  linear  Hoo  control,  the  closed  loop  system 
would  have  the  desired  induced  L2  gain  properties,  since  the  zero  dynamics  of  the  transformed 
system  are  totally  unobservable  in  the  linearized  subsystem,  and  the  output  of  interest  depends  only 
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on  the  linearized  subsystem.  This  point  turns  out  to  be  moot  for  the  example  problem,  however, 
since  the  output  z  —  X2  -  x\  yields  a  relative  degree  one  system  with  unstable  zero  dynamics,  as 
shown  below. 


A  simple  calculation,  letting  yi~h~X2—x\  yields 


Lfjh  =  [-2xi  1] 


=  1 


thus  indicating  the  system  has  relative  degree  one.  Also,  we  find 


Lah  =  [-2x1  1] 

so  that  the  linearizing  control  is 


XiX2 


—  Xi  —  2x^X2 


u  =  -j—r{-Lah  -^v)  =  2xjx2  -  xl-\-v 


In  the  transformed  coordinates  we  obtain  the  input-output  linearized  system 


yi 

V 

h 

2/2(2/!  +2/2) 

(3.51) 


(3.52) 


(3.53) 


(3.54) 

The  zero  dynamics  of  (3.54)  are  obtained  by  requiring  2/i  =  0,  =  0  so  that  v  =  0,  and  leaving 

m  =  v\  (3.55) 

From  Lemma  2.1.3  it  is  trivial  to  conclude  instability  of  (3.55). 


3.3  Recursive  Backstepping 

In  this  section,  we  apply  the  theory  of  Section  2.3  to  (3.1).  As  in  the  previous  section,  we  first 
start  with  stabilizing  designs.  We  then  seek  ways  to  tailor  the  algorithm  to  achieve  the  desired 
disturbance  attenuation  objective. 

Recall  (3.1),  and  observe  that  without  considering  the  output  this  system  is  already  in  the 
desired  configuration  for  a  backstepping  design.  Thus,  we  consider  the  first  scalar  equation  in  (3.1), 

xi  =  X1X2  (3.56) 
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for  which  the  state  variable  X2  makes  a  logical  choice  for  a  pseudocontroL  Under  recursive  backstep- 
ping,  our  objective  is  to  choose  a  suitable  desired  value  for  X2^  which  we  call  a  stabilizing  function, 
a{xi)^  such  that  if  we  can  achieve  that  value  perfectly,  then  (3.56)  will  be  stabilized.  One  logical 
choice  for  a{xi)  above  would  be 

a{xi)  —  —x\  (3.57) 

so  that  the  closed  loop  system  equation  would  be 

xi  —  (3.58) 

which  can  trivially  be  shown  to  be  GAS  by  using  the  Lyapunov  function  V{xi)  =  \x\  and  the 
arguments  of  Section  2.1.1,  or  simply  by  invoking  Lemma  2.1.3.  However,  we  recognize  that  we  will 
never  achieve  (3.57)  perfectly  by  choosing  u  in  the  second  scalar  equation  of  (3.1), 

X2  —  ~\-u  (3.59) 

so  we  introduce  the  error  variable  e  —  X2-  a{xi)  ^  X2 xj^  and  in  transformed  coordinates  obtain 

Xi  =  xi{e  —  Xi) 

x\~-  2x\  +  2x\e  (3.60) 

We  now  consider  the  globally  positive  definite  Lyapunov  function  V{xi^  e)  —  \x\  +  differenti¬ 

ating  with  respect  to  time  to  obtain  (letting  d  =  0) 

V  =  xiXi  +  ee 

~  x\{e  -  x\)  -h  e{x\  —  2x\  +  2x\e  +  u) 

=  —x\  +  e(2xl  -h  2xle  -  2x1  “k  (3.61) 

Thus,  choosing 

u  =  2x1  ~~  ~  “  --2xf  -  2xfx2  -  Cl (x2  -hXi)  (3.62) 

with  Cl  >  0  guarantees  the  closed  loop  system  is  GAS,  and  that  Xi  and  e  converge  asymptotically 
to  zero  as  time  goes  to  infinity. 
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Alternatively,  one  might  choose  the  stabilizing  function  above  as 


a{xi)^~l  (3.63) 

so  that  in  the  closed  loop  (3.56)  would  become 

=  -xi  (3.64) 

which  is  not  only  stable,  but  stable  in  the  first  approximation.  Following  the  same  procedure  as 
above,  for  this  choice  of  a{xi)  we  obtain 

V  —  xixi  +  ee 

-  xj{e  -  1)  +  e{xl  +  u) 

=  ~xl  4-  e{2x\  +  u)  (3.65) 

Thus,  now  we  could  choose  the  control 

u  =  —2x\  -  cie  —2x\  ”  Ci{x2  +  1)  (3.66) 

with  Cl  >  0  to  guarantee  the  closed  loop  system  is  GAS,  and  the  same  asymptotic  convergence 
properties  for  Xi  and  e  hold.  However,  we  must  be  careful  here,  because  since  e  =  a;2  +  1,  by  making 
(a:i,  e)  =  (0,0)  GAS,  we  have  rendered  the  equilibrium  {xi,X2)  ~  (0,  —1)  GAS  in  the  original  system 
coordinates,  which  was  not  our  objective.  This  could  have  posed  problems  if  we  had  only  been  trying 
to  obtain  local  asymptotic  stability,  so  that  in  view  of  (3.65)  we  had  chosen  u  —  —  cie  =  — ci(a;2  + 1). 
We  know  from  Section  3.1  that  such  a  linear  control  fails  to  stabilize  {xi,X2)  —  (0,0),  although 
it  does  make  (a?i,e)  —  (0,0)  LAS.  Nevertheless,  we  see  from  (3.61)  that  we  could  have  chosen  u 
according  to  (3.66)  as  a  result  of  our  first  choice  of  stabilizing  function,  simply  choosing  Ci  large 
enough  for  the  lower-order  negative  definite  terms  to  dominate  the  higher-order  cross  terms  in  a 
neighborhood  of  the  origin.  Comparing  (3.62)  and  (3.66),  we  see  that  (3.66)  is  a  much  simpler  and, 
in  general,  smaller  magnitude  control,  making  it  appear  a  more  attractive  choice. 

It  is  clear  from  this  discussion  that  recursive  backstepping  can  successfully  be  used  to  globally 
stabilize  (3.1).  The  above  application  of  the  theory  also  clearly  points  out  the  nonuniqueness  of 
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solutions  mentioned  in  Section  2.3,  and  thus  the  ensuing  need  to  exercise  sound  engineering  judgment 
in  using  this  method.  As  shown  above,  the  obvious  choice  for  the  control  is  not  always  the  best 
choice,  and  one  principle  to  keep  in  mind  is  to  avoid  unnecessary  cancellations  whenever  possible. 

Recursive  backstepping  is  aimed  primarily  at  stabilization,  and  thus  we  note  that  we  have  yet 
to  address  the  disturbance  attenuation  part  of  the  design  problem,  which  we  now  attempt  to  do. 
The  algorithm  itself  provides  no  direct  means  of  achieving  this  sort  of  objective,  and  we  therefore 
present  an  original  approach  to  solving  the  problem.  Recall  from  the  above  discussion  that  Xi{t)  and 
e{t)  are  regulated  to  zero  as  time  goes  to  infinity.  Thus,  we  might  achieve  the  desired  disturbance 
attenuation  objective  by  choosing  e{t)  to  be  equal  to  the  quantity  we  desire  to  make  small,  i.e.,  set 
e  =  z  =  X2  —xl.  We  then  get 

xi  -  xi(e 

e  —  xl  —  2xf  —  2xle  +  rt  +  d  (3.67) 

for  the  system  equations  in  x^e  coordinates.  Note  that  in  so  doing  we  are  implicitly  choosing  a 
stabilizing  function  n(a;i)  ==  which  actually  makes  (3.56)  unstable  in  the  closed  loop!  This  is  not 
a  standard  choice  of  stabilizing  function,  and  since  it  makes  the  first  equation  of  (3.67)  unstable, 
it  prevents  us  from  sequentially  building  up  a  simple,  suitable  Lyapunov  function  as  is  done  in  the 
normal  procedure.  For  if  we  choose  V  as  before  we  find 

V  —  XiXi  -h  ee 

=  Xi{e-\-  x\)  -h  e{xl  -  2xle  -  2xf  -h  u) 

=  Xi  e{2xl  —  2x^6  -  2x1  +  ‘^)  (3.68) 

and  we  see  that  no  choice  of  u  can  render  the  xf  term  negative  definite.  However,  we  see  from  (3.67) 
that  we  can  obtain  an  xf  term  in  V  by  including  an  x^e  cross  term  in  V.  Thus,  we  propose  the 
candidate  Lyapunov  function 

V{xije)  =  ^axf  4  -h  cxle,  a,  b  >  0  (3.69) 
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which  will  be  locally  positive  definite  in  a  neighborhood  of  the  origin  for  any  ceTl.  Differentiating 
and  grouping  like  terms  yields 

V  =  -2cXi  —  2hx\e  +  (a  H-  c)xf  -h  2{c  -  b)xle^  +  (a  +  b)xle  +  u{be  -h  cxf)  (3.70) 

from  which  we  see  we  need  c  >  0  for  negative  definiteness  of  the  xf  term.  We  also  see  a  third-order 
cross  term  xfe  that  we  cannot  dominate  since  we  have  no  xf  term,  leading  us  to  consider  a  control 
of  the  form 

u  =  — cie  +  dxi^  Cl  >  0  (3.71) 

Using  this  control,  (3.70)  becomes 

V  =  —2cx\  -  2bx\e  +  (a  H-  c  -f-  cd)x\  +  2(c  —  b)x\^  +  (a  +  6  +  &d!  ~  cci)xle  —  &cie^  (3.72) 

and  we  now  can  choose  coefficients  to  eliminate  the  undesirable  xfe  cross  term.  To  ensure  local 
negative  definiteness  of  (3.72),  we  want  to  choose  the  constants  such  that 

a-|-6-|-&d~cci  =  0  (3.73) 

a  -h  c  +  cd  <  0  (3.74) 


and  a,  6,  c,  and  ci  are  all  positive.  These  conditions  are  sufficient  to  guarantee  local  negative 
definiteness  of  (3.72)  because,  for  any  choice  of  coefficients  such  that  c  -  &  is  finite  and  positive,  there 
exists  some  neighborhood  of  the  origin  such  that  the  xfe^  cross  term  in  (3.72)  will  be  dominated 
by  the  and  xf  terms,  provided  they  have  negative  coefficients.  One  set  of  choices  that  meets  the 
above  conditions  is 

a  =  1,  b  =  ^,  c  =  i  d=  -10,  Ci  =  6  (3.75) 
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Using  these  coefficients,  our  final  Lyapunov  function  becomes 

V(xi,e}=^xj  +  ^e^  +  ^xfe  (3.76) 


which  has  derivative 


18 


e  — 


1  4  7  2  2 


8 


(3.77) 
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The  control  is 


u  =  — 6e  —  \Qx\  —  -~^x\  —  63^2  (3.78) 

which,  as  discussed  earlier,  is  guaranteed  to  locally  stabilize  (3.1)  and  also  cause  local  asymptotic 
convergence  of  aii  and  e  =  X2  —  x\  to  zero  as  time  goes  to  infinity. 


3,4  Nonlinear  H^o  Control 

In  this  section,  we  apply  the  state  feedback  nonlinear  ^7oo  control  theory  of  Section  2.4.3  to  (3.1). 
We  note  first  of  all  that  (3.1)  satisfies  Assumption  2.4.1,  since  the  rank  of  dz/du  =  [0  1]^  equals 
one,  which  is  the  number  of  inputs  for  this  SISO  problem.  Thus,  we  proceed  to  apply  the  method 
by  defining  the  Hamiltonian 

U  =  Vxf{x,  n,  d)  4-  z'^z  -  -  Vx^xiX2  +  Vx^{x\  u d)  {x2  ~  x\)‘^  +  (3.79) 


The  first-order  necessary  conditions  (see  2.121) 


m\ 


=  0  =  T42  - 


(i 


give  the  worst  case  disturbance  and  optimal  control,  respectively,  as 

1 


(3.80) 

(3.81) 


—  2..y2^^2 


'W*  —  ^ 


(3.82) 

(3.83) 


Substituting  (3.82)  and  (3.83)  into  (3.79),  we  wish  to  find  a  positive  semidefinite  solution  to  the 
Harailton-Jacobi  Inequality  (see  (2.125)) 

Kt  =  V^^XiX-2  +  7^2  {A  +  ^^^2)  +  A-  +  4  +  \yx2  -  ^ 


which  simplifies  to 


Vx^XiX-2  +  +  A  -  2a.fa;2  +  -  1)  <  0 


(3.85) 
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If,  as  in  Section  3.3,  we  let  A  —  —  1,  then  (3.85)  can  be  written 

Va:^XiX2  +  ^2  “  +  xj  +  <  0  (3.86) 

If  we  propose  a  solution  of  the  form  V  —  ax^  +  a,  6  >  0,  we  find  (3.86)  becomes 

xl  +  (2a  4  2b  —  2)a;ia;2  4-  (1  +  ^  0  (3.87) 

and  we  see  that  we  can  not  solve  (3.87)  in  a  neighborhood  of  the  origin  due  to  the  positive  coefficient 
on  xf.  We  therefore  propose  a  candidate  solution  of  the  form  V  =  axf  H-  hx2  +  cxlx2^  a,  b  >  0,  for 
which  (3.86)  becomes 

(1  -h  c  4  ^)xt  4  2cxlxl  4  (2a  4  2b  -  2  4  Xbc)xlx2  4(14  Ab^)^;!  <  0  (3.88) 

From  the  coefficient  we  clearly  must  have  A  <  0  in  order  to  ensure  local  solution  of  (3.88),  which 
implies  7  >  1.  Thus,  as  in  Section  3.3,  we  will  consider  two  cases:  a  near-optimal  case  where 
A  =  —  corresponding  to  7^  =  and  a  more  suboptimal  case  where  A  =  corresponding  to 
7^  =  2.  Local  solutions  to  (3.88)  are  not  unique,  consisting  of  any  set  of  coefficients  such  that 

2a  4  2b  -  2  4  Abe  =  0  (3.89) 

I4c4^  <0  (3.90) 

1  4  Ab^  <  0  (3.91) 

and  a  and  b  are  positive.  For  A  ”7^5  one  such  set  of  coefficients  is 

a=  16,  b:-  10,  c  =  500  (3.92) 

and  we  note  that  the  large  positive  c  term  is  required  to  satisfy  (3.89)  and  (3.90)  ((3.89)  requires  c  to 
be  positive,  and  c  =  400  is  the  smallest  positive  c  satisfying  (3.90)).  The  corresponding  expressions 
for  y,  74+ ,  and  the  control  u  are 

V  =  l^xj  4  lOa’2  4  500x1x2 

=  -Uixj  4  lOOOxjxl  -  AAxl 

u  =  -10a:2  -  250xj  (3.93) 
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For  A  =  one  such  set  of  coefficients  is 

a  =  16,  &  =  10,  c  =  10  (3.94) 

and  we  note  that  we  do  not  need  nearly  as  large  a  c  term  to  satisfy  (3.89)  and  (3.90).  The  corre¬ 
sponding  expressions  for  F,  H*,  and  the  control  u  are 

V  =  \^x\  +  10a;2  +  l^x\x2 

-h  2{^x\x\  -  AQx\ 

u  =  -10a;2  “  ^^1  (3.95) 

Comparing  (3.93)  with  (3.95),  we  see  that  the  near-optimal  solution  requires  much  greater  control 
effort  in  terms  of  gain  on  x^,  as  we  might  expect  since  we  are  trying  to  achieve  a  higher  level  of 
disturbance  attenuation.  However,  the  increase  in  disturbance  attenuation  (reduction  in  induced  L2 
gain)  is  only  about  a  factor  of  two,  while  the  required  gain  increase  is  near  a  factor  of  fifty.  We  thus 
seem  to  be  seeing  the  familiar  type  of  high  bandwidth  phenomenon  we  see  in  linear  Hqo  control  as 
we  approach  the  optimal  solution. 

For  later  comparison  purposes,  it  is  also  of  interest  to  compute  the  solutions  to  the  Hamilton- 
Jacobi-Issacs  equations  for  the  above  cases.  This  involves  solving  (3.90)  and  (3.91),  in  addition  to 
(3.89),  with  equality.  For  A  =  the  solution  is 

a-:  11,  5=10,  c  =  400  (3.96) 


giving 

V  =  llxl  +  100^2  +  ^^^x\x2 

7i^^O{x^) 

~Wx2  -  200a;?  (3.97) 

while  for  A  =  - 1 ,  the  solution  is 

a  =  2.732,  5  =  \/2,  c  =  8.90  (3.98) 
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giving 

V  =  2.732xj  +  V^a;2  +  8.9a;ia;2 

n*  =  o(x^) 

u  —  —\/2xi  —  4.45xf  (3.99) 

For  both,  cases,  we  ensure  V  is  zero  through  third  order,  and  in  fact,  the  fourth-order  term  that 
makes  V  positive  is  2cxiX2. 

3.5  SDRE  Techniques 

Herein  we  apply  the  SDRE  nonlinear  regulator  theory  to  (3.1)  in  Section  3.5.1,  and  the  SDRE 
nonlinear  Boo  control  theory  to  (3.1)  in  Section  3.5.2. 


3.5.1  Nonlinear  Regulation  Via  the  SDRE  Method 

For  this  example  we  shall  choose  to  minimize  the  infinite-horizon  cost  function 

minimize  J  =  -  /  [x^Q{x)x  +  u^R{x)u] 

2  Jto 

with  R{x)  =  and 

Q{x)  = 


dt 


(3.100) 


IJ?  0 

=  H^{x)H{x)  — 

/i  0 

0 

0  1 

0  1 

0  1 

so  that  ^  allows  us  to  set  a  relative  weight  on  state  deviations  from  zero,  and  (3.100)  is  guaranteed 
convex  since  Q  and  R  are  positive  definite.  The  scalar  weights  p  and  p  will  be  treated  as  design 
parameters,  the  effects  of  which  will  be  observed  later. 

Recalling  that  we  need  to  factor  the  drift  term  in  (3.1)  as  a(a’)  —  A{x)x^  we  define 


Ai{x)  ^ 


X2  0 

0 


,  ^2(2;)  = 


0  Xi 
xi  0 


(3-101) 
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and  observe  that  (recalling  Section  2.5.1) 


ax2  (1  ”  oi)xi 

A(a,  x)  —  aAi{x)  +  (1  -  a)A2(x)  —  (3.102) 

xi  0 

is  a  valid  SDC  parametrization  for  all  a  eTZ,  In  fact,  it  can  easily  be  verified  that  (3.102)  gives  all 
valid  SDC  parametrizations  of  a{x)^  and  Ai  corresponds  to  a  =  1,  while  A2  corresponds  to  a:  ~  0. 
Recalling  that  we  desire  controllable  SDC  parametrizations,  we  compute  controllability  matrices  for 
Ai,  A2,  and  A{ol^x)  and  respectively  obtain 

0  0  ^  Xi  0(1  —  a)xi 

MM=  ,MM=  (3-103) 

10  10  10 

so  that  Ai  is  an  uncontrollable  parametrization,  and  A2  and  A{ol^x)  are  controllable  so  long  as 

^  0.  We  note  the  similarity  between  the  lack  of  existence  of  a  controllable  SDC  parametrization 

for  0)1  =  0  to  the  lack  of  a  well-defined  relative  degree  for  (3.1)  at  a)i  =  0  observed  in  Section 

3.2,  and  just  as  in  that  section,  proceed  with  the  SDRE  nonlinear  regulator  algorithm  in  hopes  of 

obtaining  a  stabilizing  control  in  spite  of  technically  failing  to  meet  the  controllability  assumption. 

We  also  note  that  by  our  choice  of  H{x)^  we  have  guaranteed  observability  of  (iJ,  A(a,  a)))  for  any 

SDC  parametrization  A(a,  a;). 


Equation  (3.105)  yields  three  nonlinear  scalar  equations 


2a:iPi2  -  ^  +  =  0 

(3.106) 

2xipi2  -  %  +  1  =  0 

P 

(3.107) 

{pn+P22)xi  ^2 

(3.108) 

only  the  first  two  of  which  need  to  be  solved  to  obtain  the  control 


u  =  —R  Px 


1  ^11 

Pl2  P22 

Equations  (3.106)  and  (3.107)  can  be  solved  sequentially  to  give 


-  “^(Pi2^i  +P22a?2)  (3,109) 


Pi2{xi)  =  p’^xi  ±  p{p^xl  +  /i^)^  (3.110) 

P22{xi)  =  p(l  +  2a?ipi2)^  (3.111) 


where  we  have  taken  the  positive  square  root  of  ^22(^1)  in  (3.111)  to  obtain  a  positive  definite  p, 
and  have  left  the  sign  of  the  square  root  in  (3.110)  undetermined  since  we  don’t  know  which  sign 
produces  a  stabilizing  solution.  To  determine  if  a  stabilizing  solution  to  (3.104)  exists,  we  expand 
P12  and  P22  in  a  Taylor  series  about  =  0  (through  first  order)  to  obtain 

Pi2(^i)  -  ±PP  +  p’^xi  +  •  •  •  (3.112) 

P22(a?i)  =  p  +  p(l  ±  p)xi  +  •  •  ■  (3.113) 


Using  these  expressions  we  obtain  a  second-order  approximation  for  the  control 

P  1  2  .  N 

u  =  =F— - ^2  “  - (1  ±  p)xiX2 

p  p  p 

which,  upon  substitution  into  (3.1),  gives  the  closed  loop  system  equations 


Xi 

X1X2 

^2 

Tp  2:1  -  jX2  -  i(l  ±  p.)xiX2 

(3.114) 


(3.115) 
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and  corresponding  solutions  of  (3.107) 

=  P5  p(l  +  if?x\)  2  (3.120) 

The  first  set  of  solutions  to  (3.119)  and  (3.120)  yield  the  control 

1 

U  = - X2 

P 

which  is  clearly  not  stabilizing  since  it  is  linear.  The  second  expression  for  P22  in  (3.120)  can  be 
expanded  in  a  Taylor  series  about  =  0  to  second  order  (the  second-order  coefRcient  is  zero) 

p22{xi)^p  (3.121) 
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so  that  to  second  order  the  control  becomes 


u  =  — X2  -  2x1 
P 


(3.122) 


The  control  (3.122)  gives  a  resulting  closed  loop  system 


- 

- 

“ 

" 

- 

Xi 

0 

0 

Xi 

-f 

XiX2 

X2 

0 

_i 

p 

X2 

.  . 

Again  using  center  manifold  theory,  it  can  be  determined  that  the  xi  equation  becomes 


(3.123) 


xi  —  -pxl  +  0{xl) 


(3.124) 


which  from  Lemmas  2.1.2  and  2.1,3  we  may  conclude  to  be  locally  stable  for  any  positive  value  of 
p.  Thus,  the  SDRE  nonlinear  regulator  does  indeed  provide  a  locally  stabilizing  control  for  (3.1), 
provided  that  our  linearly  uncontrollable  state  is  also  linearly  unobservable  in  the  cost  function. 
This  phenomenon  of  the  SDRE  method  stabilizing  uncontrollable  (in  the  sense  of  not  having  a 
controllable  SDC  parametrization)  modes  is  very  interesting,  and  was  also  observed  by  Parrish  in 
[53]. 

We  note  that  in  the  final  control  (3.122),  the  gain  on  the  controllable  X2  state  increases  directly 
in  proportion  to  a  decrease  in  the  penalty  on  control  usage,  p,  just  as  we  might  expect  from  LQR 
theory.  Also,  if  the  state  weighting  on  X2  is  changed  from  1  to  P,  the  control  becomes 

u=-^x2-2xI  (3.125) 

SO  that  now  the  gain  on  the  controllable  state  is  determined  by  the  ratio  and  control  usage  will 
increase  proportionately  with  increasing  penalty  on  state  deviations. 


3,5,2  Nonlinear  Hoo  Control  Via  the  SDRE  Method 

We  now  proceed  to  apply  the  full  state  feedback  nonlinear  Hoo  control  theory  of  Section  2.5.2  to 
(3.1).  We  first  need  to  establish  that  (3.1)  meets  the  required  assumptions.  Recalling  our  convention 
for  the  system  equations 

X  =  a{x)  H-  b{x)u  +  g(x)d 
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;2;  =  k{x)  +  di2{x)u 


d(2{x)[hix)di2(x)]  =  {01]  (3.126) 

we  identify  a,  6,  and  g  as  before  and  h{x)  =  [x2  —  xj  0]^,  di2  =  [0  1]^,  so  that  indeed  a(0)  =  0, 
/i(0)  =  0,  and  the  last  equation  in  (3.126)  is  satisfied.  Next,  we  establish  our  SDC  parametrizations 
of  (3,126)  as 


a{x)  —  A%{x)x 

h{x)  =  H{x)x  (3.127) 


where 


as  in  Section  3.5.1,  and  we  take 


A2{x)  ^ 


0  xi 
xi  0 


H{x) 


—Xi  1 
0  0 


(3.128) 


(3.129) 


noting  that  (3.129)  is  the  unique  SDC  parametrization  for  h.  With  these  choices  of  A^ix)  and  H{x), 
we  find  that 


j42(0)  = 


0  0 


(3.130) 


H{0)  = 


0  0 

0  1 


(3.131) 


giving  an  observability  matrix 


Ma 


0  0 

0  1 
0  0 


(3.132) 


0  0 
0  0 


which  is  clearly  rank  one.  Since  both  eigenvalues  of  j42(0)  are  zero,  (3.132)  allows  us  to  conclude 
that  the  linearization  of  (3.126)  is  not  detectable,  and  thus  the  SDRE  nonlinear  Hoo  solution  method 
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is  not  guaranteed  to  be  LAS  for  this  system  by  the  standard  theory  involving  LAS  linearizations. 
Since  the  SDRE  must  first  be  solved  for  either  of  the  methods  of  Section  2.5.2,  we  first  attempt  the 
second  approach.  Then  we  attempt  to  apply  the  first  technique,  to  see  if  it  works  and  to  give  insight 
into  reasons  for  failure  if  that  is  the  end  result  . 

We  need  to  solve  the  state  feedback  nonlinear  Hoo  SDRE 

A'^{x)P  +  PAix)  -  P[B{x)B'^{x)  -  ^Gix)G'^(x)]P  +  H^ix)H{x)  =  0  (3.133) 


which  becomes  after  substitution  of  the  proper  values 


0  xi 
xi  0 


Pll  Pl2 
Pl2  P22 


+ 


Pn  Pi2 
Pl2  P22 


0  Xi 

Xi  0 


_ 

J  L 

P 

J 

L 

J 

L 

- 

Pll 

P12 

0 

0 

Pll 

P12 

+ 

-Xi 

0 

P12 

P22 

0 

1 

P12 

P22 

1 

0 

-Xi  1 

0  0 


0  0 
0  0 


(3.134) 


Again  defining  A  =  —  1^ ,  (3.134)  simplifies  to 


Xi 


2pi2  Pn  +P22 
Pll  +  P22  2pi2 


+  A 


P12  P12P22 


P12P22  P22 


+ 


—Xi  ~Xi 

—Xi  1 


0  0 
0  0 


(3.135) 


Equation  (3.135)  yields  three  nonlinear  scalar  equations 

2xipi2  d-  Apf2  +  =  0 

2Xipi2  +  ^P22  +  1  =  0 
{pn  +P22)Xl  +  Api2P22  -Xi=0 

only  the  first  two  of  which  need  to  be  solved  to  obtain  the  control  (2.171) 


(3.136) 

(3.137) 

(3.138) 


u  -r  -b^Px  =  -[0  1] 


X  =  -pi2Xi  -P22X2 


Pll  P\2 
Pl2  P22 

Equations  (3.136)  and  (3.137)  can  sequentially  be  solved  analytically  to  give 


(3.139) 


Pi2(a;i)  =  y(-l  i  v^l  -  •^) 


P22(2;i)  = 


2x^ 


(3.140) 

(3.141) 
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where  we  have  taken  the  positive  square  root  to  obtain  (3.141),  and  have  left  the  sign  of  the  square 
root  as  yet  undetermined  in  (3.140).  This  sign  will  be  chosen  to  yield  a  stabilizing  solution.  We 
note  here  that  (3.136)  has  been  solved  exactly,  but  although  (3.137)  appears  to  also  have  been 
solved  exactly,  it  actually  has  not.  This  is  because  p\2  is  linear  in  a^i,  so  that  if  we  expand  P22  = 
Po  T  +  'P2^\  +■•*•?  the  first-order  equation  in  Xi  resulting  from  (3.137)  implies  pi  =  0.  Thus 
there  is  no  way  to  satisfy  the  second-order  equation  in  xi .  To  determine  if  a  stabilizing  solution  to 
(3.134)  exists,  we  use  the  Taylor  series  expansion  of  P22  about  a;i  =  0  to  first  order  to  obtain 


P22(a^i)  ^ 


(3.142) 


We  then  define  ki  =  — 1  — Vl  “  \  ^2  =  —  1  +  a/1  -  A,  and  use  these  expressions  to  get  a  second-order 
approximation  for  the  control 

(3.143) 

where  we  take  z  =  1  or  2  depending  on  which  sign  of  the  square  root  term  in  pu  we  wish  to  use. 
If  we  recall  all  other  stabilizing  controls  we  have  obtained  in  this  chapter,  we  observe  that  each  one 
was  of  the  general  form  u  —  ax\  +  hx2 ,  where  a  and  h  were  both  negative.  In  fact,  we  know  b  must  be 
negative,  and  it  is  easily  proven  using  center  manifold  theory  that  we  must  have  a  <  -1  to  obtain  a 
stable  closed  loop  system.  Using  this  insight,  we  should  expect  that  using  ki  in  (3.143)  would  give 
the  stabilizing  solution,  and  we  therefore  proceed  under  this  assumption.  For  A  =  “  obtain 

the  control 

u  =  -200x1  -  10x2  (3.144) 


whereas  for  A  =  we  obtain 


u  =  -4.45a?f  —  V2x2 


(3.145) 


and  recalling  (3.97)  and  (3.99),  we  see  that  the  above  matches  our  earlier  solutions  from  the 
Hamilton- Jacobi- Issacs  equations  exactly.  Now,  even  though  we  know  (3.144)  and  (3.145)  are  sta¬ 
bilizing  from  other  sources,  we  would  like  to  know  if  this  technique  provides  us  a  locally  positive 
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definite  Lyapunov  function  to  establish  closed  loop  stability  on  its  own.  Recalling  that  we  have 

V^{x)  =  2x^ P{x)  =  2a;ipii  +  (3.146) 

we  need  to  solve  simultaneously  the  integral  equations 

^  -  j  dxi  (3.147) 

“  /  ~  (3.148) 

Equation  (3.147)  can  be  solved  to  give 

V{x)  =  Piix\  -h  ^x\x2  +  1{X2)  (3.149) 

and  (3.148)  can  be  solved  to  give 

V(a))  =1  -h  +  9{^\)  (3.150) 

so  that  we  must  have  f{x2)  =  \J^^2  9[^i)  —  Pii^i*  However,  there  is  still  a  factor  of  two 

discrepancy  between  the  x\x2  coefficients  in  the  above  two  expressions  for  V.  Now,  recall  that  we 
have  assumed  P{x)  to  be  symmetric,  but  there  is  really  no  need  to  enforce  this  assumption  if  we 
are  solving  the  SDRE  analytically.  Careful  examination  of  the  Hamilton-Jacobi-Issacs  equation  for 
this  problem  with  this  symmetry  assumption  removed  leads  to  three  scalar  equations.  Two  of  these 
equations  are  the  same  as  (3.136)  and  (3.138),  while  (3.137)  becomes 

^X\P2\  "h  ^^22  +  1=0  (3.151) 

where  the  only  change  is  that  pi2  has  become p2i  in  (3.151).  Thus,  there  is  no  symmetry  requirement 
on  P{x)^  and,  in  fact,  to  assure  that  Vx{x)  —  2x^P{x)  has  a  consistent  solution  V,  we  can  set  p2i  ~ 
2pi2.  We  note  this  does  not  afifect  the  optimal  control  derived  previously,  since  for  nonsymmetric 
P,  the  optimal  control  is  in  fact  given  by 

w*  =  -h^P'^X  =  -[pi2  P22]x 
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and  the  first-order  solution  for  P22  does  not  change  under  this  modification.  Making  this  change  we 
obtain  the  Lyapunov  function 


■r  r  /  \  9  2/^1  O 

^  Pn^l  +  —XIX2  + 


(3.152) 


which  does  indeed  solve  the  required  set  of  partial  differential  equations.  Now,  (3.138)  can  be  solved 
to  first  order  to  give  pn  1  -h  and  thus,  since  A  <  0,  (3.152)  is  locally  positive  definite, 

and  we  can  indeed  conclude  that  the  origin  in  the  closed  loop  system  is  LAS.  In  fact,  when  the 
appropriate  values  of  A  are  substituted,  (3.152)  agrees  exactly  with  the  Lyapunov  functions  found 
in  Section  3.4,  by  making  the  associations  a  =  pn,  b  =  ^ad  c  =  We  also  point  out  that 

V  in  (3.152)  can  be  written  in  the  form  V  -  x'^W{x)x,  where  W(0)  is  a  positive  definite  matrix, 
and  the  off-diagonal  elements  of  W{x)  are  nonunique,  so  that  W  may  or  may  not  be  symmetric. 

Now,  to  apply  the  first  approach  to  SDRE  nonlinear  Hqo  control,  we  also  need  to  compute  the 
contribution  to  the  control  from  the  Mp  term  we  previously  defined  as 


Mp  -  (3.153) 

ox 

which  is  given  by  um  =  -^h^MpX,  Using  the  product  rule  -^{Px)  =  P  +  =  P  +  Afp,  an 

expression  for  the  matrix  function  Mp  can  be  derived  without  having  to  invoke  tensor  operations. 
The  resulting  expression  for  the  ijth.  entry  of  Mp  is 


Mpij  —  ^  ^  Xk 


k=i 


dPik 

dxj 


Using  the  symmetric  first-order  solution  to  the  Poo  SDRE,  (3.133), 


(3.154) 


1  “  i 

Mp  can  be  computed  as 


(3.155) 


[  0  j 

so  that  “  0,  producing  the  same  control  as  in  both  previous  methods.  Also,  we  find  that  the 
second  bracketed  term  in  (2.173)  reduces  to  2^x\x\,  while  our  inability  to  solve  (3.137)  exactly  also 
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leaves  a  term  equal  to  2^x\x2^  Thus,  our  solution  to  the  Hamilton-Jacobi-Issacs  equation  using 
this  approach  is  again  zero  to  third  order,  with  the  next  remaining  term  equal  to  A^x\x2  —  2cx\x2> 


We  obtain  our  Lyapunov  function  from  this  approach  via 


V{x)  =  x^P{x)x  —  x^ 


Pn 


2  Z/Cl  2  /“J-  ■ 

X  =  pnx^  -h  -y^x^X2  +  \  -r-x 


(3.156) 


that  in  fact  we  have  obtained  the  same  solution  to  the  nonlinear  local  Hqq  control  problem  using 


all  three  approaches. 

Summarizing,  it  appears  that  both  SDRE  approaches  may  be  used  to  solve  the  local  state  feed¬ 
back  nonlinear  Hoo  control  problem  analytically.  Using  the  first  approach  of  Section  2.5.2,  we  propose 
V  =  x'^P(x)x,  P  =  so  that  Vx  =  x^{2P  Mp),  and  we  must  solve  the  Hoo  SDRE  (2.161)  plus 
the  additional  inequality  from  the  second  bracketed  term  in  (2.173).  The  advantages  of  this  method 
are  that  the  symmetry  of  P  make  the  solution  of  (2.161)  simpler,  and  that  we  are  guaranteed  the 
existence  of  a  locally  positive  definite  Lyapunov  function  of  known  form  under  certain  known  condi¬ 
tions.  The  main  disadvantage  is  having  to  compute  Mp,  and  dealing  with  the  second  bracketed  term 
in  (2.173).  Using  the  second  approach  of  Section  2.5.2,  we  propose  Vx  =  2rr^P,  and  we  must  solve 
the  Hoo  SDRE  (2.161),  followed  by  the  n-dimensional  set  of  partial  differential  equations  dictated 
by  Vx  for  the  Lyapunov  function  V.  The  advantage  here  is  we  only  need  to  solve  the  state  feedback 
Hoo  SDRE,  and  in  the  limit  as  7  approaches  infinity,  (2.161)  approaches  the  appropriate  Hamilton- 
Jacobi-Bellman  optimal  control  equation.  The  main  disadvantage  is  that  we  can’t  as  yet  guarantee 
the  existence  of  a  locally  positive  definite  Lyapunov  function,  and  we  must  solve  n  pde’s  to  obtain 
a  candidate.  In  this  example  problem  we  were  able  to  find  such  a  Lyapunov  function  successfully, 
but  it  required  taking  a  nonsymmetric  solution  Pu  to  the  state  feedback  Hoo  SDRE,  (2.161).  In 
this  case,  the  solution  was  underdetermined,  and  P21  was  chosen  in  an  ad  hoc  manner,  based  on 
what  we  knew  the  answer  should  be  to  obtain  a  consistent  Lyapunov  function.  Thus,  it  may  not 
always  be  straightforward  to  construct  the  appropriate  nonsymmetric  SDRE  solution.  In  doing  so 
for  this  example  problem,  we  obtained  equal  expressions  for  Vx  and  hence  V  using  both  methods. 
This  implies  x^{2P-u)  =  x'^(2P  +  Mp)^  and  yet  from  simple  calculation  we  find  2Pu  ^  2P  Mp. 
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This  observation  suggests  that  determining  more  specific  conditions  under  which  both  methods  may 
be  used  to  obtain  the  same  result  may  be  difficult  to  accomplish.  Nevertheless,  this  topic  is  pursued 
in  Section  4.5,  and  partial  results  are  obtained  relating  the  two  solution  approaches  for  the  case  of 
symmetric  P. 

A  final  comment  on  the  SDRE  nonlinear  Poo  control  techniques  is  that  we  have  not  addressed  the 
effects  of  diflferent  SDC  parametrizations  on  solvability  of  (2.161)  and  (2.173).  Using  the  uncontrol¬ 
lable  SDC  parametrization  Ai(a;),  it  is  straightforward  to  solve  (2.161)  and  show  that  no  stabilizing 
solution  exists.  Recalling,  A[a,x)  =  aAi(x)~\-  (1  -  a)A2{x),  we  have  thus  solved  the  problem  for 
a  ==  1,  0,  respectively  corresponding  to  the  solutions  for  Ai  and  A2.  For  any  value  of  a  other  than 
1  or  0,  the  three  nonlinear  scalar  equations  resulting  from  (2.161)  become  coupled,  and  cannot  be 
solved  sequentially.  Thus,  it  is  hard  to  determine  analytically  the  effects  of  a  on  solvability  of  the 
example  problem.  Using  a  diflFerent  value  of  a  and  hence  a  different  SDC  parametrization  of  a,  it 
may  be  possible  to  solve  (2.161)  to  higher  than  third  order,  perhaps  even  exactly.  This  research  has 
not  been  carried  out,  however,  as  the  more  fruitful  line  of  research  of  numerical  solution  approaches 
has  been  pursued. 

In  this  chapter  we  have  applied  the  nonlinear  control  design  methods  of  Chapter  2  to  a  moti¬ 
vational,  second-order  example  problem.  In  doing  so,  we  have  illustrated  the  principles,  strengths, 
and  weaknesses  associated  with  each  method,  allowing  us  to  see  where  the  SDRE  methods  may 
offer  advantages.  The  example  problem  also  raises  a  number  of  questions  about  SDRE  solution 
approaches,  which  will  be  addressed  in  Chapter  4. 
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IV.  Insights  into  Design  Issues 


In  this  chapter  we  address  several  issues  concerning  SDRE  control  algorithms  that  were  introduced 
in  Chapters  2  and  3,  These  issues  are  primarily  related  to  two  main  thrusts:  the  relationship 
between  the  suboptimal  SDRE  regulator  and  the  corresponding  optimal  controller  for  the  infinite 
time  problem,  and  solution  approaches  to  HJIs  via  SDREs.  With  respect  to  the  former,  we  show 
how  convexity  affects  sufficient  conditions  for  local  and  global  optimality,  we  derive  and  interpret  a 
simplified  necessary  condition  for  the  SDRE  method  to  give  the  optimal  solution,  and  we  develop 
a  numerical  algorithm  to  enforce  satisfaction  of  the  simplified  necessary  condition.  In  the  latter 
regard  we  consider  a  number  of  varied  topics:  two  proposed  solution  approaches,  their  Lyapunov 
functions  and  corresponding  relationships,  symmetry  of  SDRE  solutions,  numerical  versus  analytical 
SDRE  solutions,  necessary  conditions  for  solvability,  and  strict  inequality  versus  equation  problem 
formulations. 

4.1  Partial  Derivatives  of  Vector  Matrix  Products  with  x  Dependency 

Quite  often  in  the  course  of  derivations  and  proofs  involved  with  SDRE  control,  it  is  necessary  to 
compute  partial  derivatives  with  respect  to  x  of  vector  matrix  products  having  explicit  x  dependency 
(see  Sections  2.5.1  and  2.5.2  for  example).  Since  formulas  for  such  are  not  readily  available  in  texts, 
original  derivations  of  the  needed  formulas  are  given  in  this  section.  Suppose  we  wish  to  compute 
where  z(x)  e'R'^  is  a  vector  valued  function  of  x  and  P{x)  e  is  similarly  a  matrix  valued 

function  oi  x.  If  we  let 

rrP  =  z^P  (4.1) 

then  nP{x)  G  is  a  (row)  vector  function  of  x  with  elements 

n 

rui^^ZkPki  (4.2) 

k=l 
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Thus,  using  standard  convention  for  the  partial  derivative  of  a  row  vector  with  respect  to  a  vector, 


we  have 


dz^P  __  drmJ 
dx  dx 


ami 

dm2 

dm-n. 

dxi 

dx\ 

dx\ 

dm\ 

dm2 

dm„. 

dX2 

dX2 

dX2 

dm2 

dm-n. 

dxji 

dxn 

dXn 

(4.3) 


and  we  see  F  is  a  matrix  function  of  a)  as  expected.  Using  (4.2)  and  the  repeated  index  convention 


to  represent  summation  over  the  repeated  index  (akbk  =  J2k  ^kbk),  we  find 


Y 


^Pkl  +  Zk^  I^Pk2  +  Zk^ 


^Pkl  +  Zk^^  ^Pk2  +  Zk^S^ 


^.Pkl+^k^^  ltPk2  +  Zk^ 
SO  that  the  ijth  element  of  Y  can  be  written 


dxi  ^  dxi 

^Pu  4-  zu 
dX2  ^  dX2 


-L  y, 

dXji  ^  dXn 


V- 

p  ZJ 


dzk 

dxi 


Pkj 


dPkj 

dxi 


Thus,  by  observing 


and  adopting  the  convention 


it  can  be  seen  that 


dxi^kl 

.  .  ^  p, 

Qx^-^kn 

dX2 

§^^k2  ■ 

dzh  p 
dX2'^^’^ 

dzk  p 
ax„-^ki 

§^Pk2  • 

dzh  p 
dx-n 

^k^ 

^k^  ■■ 

,  .  y,  dPkn 
dxi 

^kWt 

^  dX2 

•  ■  Zk^p^ 

^  dX2 

*• 

y, 

dXn 

dz'^P  dz'^  ^  T^dP 

- - - p  +  z  — 

dx  dx  dx 


(4.4) 


(4.5) 


(4.6) 


(4.7) 


(4.8) 
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In  the  notation  in  (4.7)  we  consider  a  single  object,  and  note  that  the  Xi  with  respect  to  which 
the  partial  is  being  taken  remains  the  same  in  each  row  of  the  matrix.  This  corresponds  to  standard 
notation  in  that  we  are  taking  the  partial  of  a  row  vector,  as  indicated  by  the  premultiplying  z'^. 
By  letting  ^  =  a;,  it  is  clear  from  (4.8)  that 


dx^P  ^  rdP 

^  =  +  "  8? 

By  simply  substituting  for  P  above  and  carrying  out  the  same  operations,  we  find 

dz^P^  dz^  rp  dP^ 

dx  dx  ^  dx 

which  in  the  case  oi  z  =  x  simplifies  to 

dx^P'^  j^dP'^ 

dx  ~  dx 


(4.9) 


(4.10) 


(4.11) 


We  can  perform  similar  operations  when  P  is  post  multiplied  by  2  to  obtain 


dPz  _ 
dx 

^dz  dP 

:  P - 1 - 2 

dx  dx 

(4.12) 

where  we  have  adopted  the  convention 

1 _ 

■■■ 

dx2 

y  9Pnk. 

II 

Zi, 

dxi 

dX2 

(4.13) 

axj 

dX2 

When  z  —  X  (4.12)  simplifies  to 

dPx 

dx 

0 

=  P  +  —X 

dx 

(4.14) 

For  completeness  we  also  have 

dP^z  dP^ 

dx  dx  dx 


(4.15) 


which  simplifies  to 

dP^x  rp  dP^ 


(4.16) 
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in  the  case  where  z~x.  Note  that  in  adopting  these  conventions,  we  have  preserved  the  conventions 


that 


and 


dni^  ( dm^ 
dx  V  ^2;  y 


(4.17) 


[ab)^  —  iF 


(4.18) 


Note  that  (4.18)  even  applies  to  the  constructs  in  (4.13)  and  (4.7),  so  that  for  example 


<  rdPy  dP'^ 

dx)  ~  dx  ^ 


(4.19) 


To  conclude  this  section  we  relate  the  definitions  given  here  to  the  notation  of  Sections  2.5.1  and 
2.5.2.  Prom  (2.166)  we  see  that  Mp  in  Section  2.5.2  is  given  by  (4.13),  with  the  substitution  z  =  x, 
while  from  (4.19)  we  see  that  Mp  is  given  by  (4.7)  with  the  same  substitution.  Simple  calculations 
verify  that  for  any  symmetric  matrix  P 


dP 


Mp  —  MpX  = 

dx 


(4.20) 


so  that  x^PxX  in  Section  2.5.1  may  equivalently  be  replaced  by  any  of  the  expressions  in  (4.20). 


4.2  Convexity  Concerns  in  SURE  Nonlinear  Regulation 

In  considering  existence  and  determination  of  optimal  solutions  to  the  constrained  minimization 
problem  of  Section  2.5.1,  the  issue  of  convexity  arises.  For  example,  necessary  and  sufiicient  condi¬ 
tions  for  global  optimality  are  most  easily  formulated  for  convex  optimization  problems  [46] .  Even 
in  seeking  local  optima,  convexity  plays  a  role  in  determining  sufficient  conditions.  In  Sections  2.5.1 
and  4.4,  the  first-order  necessary  criteria  for  optimal  solutions  to  the  constrained  control  problem  are 
given.  To  obtain  sufficient  conditions  for  a  strong  local  minimum,  the  additional  Legendre-Clebsch, 
Weierstrauss,  and  Jacobi  (nonconjugate  point)  conditions  may  be  enforced,  provided  the  system 
is  normal  [8].  It  turns  out  that  the  Legendre-Clebsch  and  Weierstrauss  conditions  are  related  to 
convexity  of  the  Hamiltonian 

Ji  —  x'^Q(x)x  -h  u^R{x)u  +  {a{x)  -f  b(x)u)  (4*21) 
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with  respect  to  the  control  u.  These  two  conditions  are  satisfied  if 

Hun  =  R{x)  >0yxeTl^  (4.22) 

which  we  always  satisfy  by  assumption  in  our  choice  of  R.  Thus,  if  we  choose  R{x)  >  0  for  all  x  and 
satisfy  the  SDRE  Necessary  Condition  for  Optimality,  we  need  only  satisfy  the  Jacobi  and  normality 
conditions  to  assure  obtaining  a  locally  minimizing  control  with  respect  ^to  the  cost  funtion  /. 

On  the  other  hand,  global  minimizations  will  rely  on  convexity  of  H  with  respect  to  x  and  u.  It 
is  therefore  of  interest  to  know  what  kinds  of  state  weighting  matrix  functions  Q{x)  yield 

^^>0Vx  (4.23) 

OX^ 

Using  the  insight  of  (4.7)  and  (4.13),  we  have  the  following  theorem,  which  provides  a  sufficient 
condition  for  convexity  of  the  state  component  of  H. 

Theorem  4.2.1  Consider  the  scalar  function  I  —  x'^Q{x)x.  Choose  Q{x)  ~  Qq  +  Qi{x)  where  Qo 
is  any  symmetric  positive  definite  matrix  and  Qi{x)  is  equal  to  a  diagonal  matrix  function  Qi{x)  — 

diag{qi{xi), ..  ^^qn{Xn))}  where  each  qi  takes  the  form  qi{xi)  =  cio  +  Ci2x'f  -h  Ci4^xj  H - -h  Cis-xf , 

with  Cij  >0,  y  =  0,  2, 4, ...  Si.  Then  I  is  globally  convex  with  respect  to  x. 

Proof:  Straightforward  application  of  either  (4.8)  and  (4.7)  or  (4.12)  and  (4.13),  followed  by 

(4.20).  ■ 

Note  that  Theorem  4.2.1  justifies  the  logical  choice  for  state-dependent  penalty  weightings  on  the 
states  which  yield  only  sums  of  terms  consisting  of  nonnegative  coefficients  multiplying  higher  order, 
even  powers  of  individual  states  in  the  cost  function.  Thus,  if  Q{x)  is  chosen  as  per  Theorem  4.2.1, 

each  state  Xi  has  a  term  in  the  cost  function  of  the  form  +  Ci2x1  -i-c^xf  H - where 

all  the  Cij  are  greater  than  or  equal  to  zero.  Such  a  state  weighting  provides  a  steeper  penalty  for 
nonzero  state  deviations  far  from  the  origin  than  purely  constant  state  weightings  does,  and  thus  can 
be  expected  to  increase  control  gains  far  from  the  origin.  We  close  this  section  by  stating  that  most 
state  weighting  matrix  function  examples  in  this  dissertation  are  chosen  to  satisfy  the  requirements 
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of  Theorem  4.2.1  so  as  to  obtain  the  desirable  properties  of  convex  functionals.  In  some  cases  we 
choose  Q[x)  to  satisfy  Ixx  ^  0,  so  that  we  have  convexity  on  a  reduced  part  of  the  state  space. 

4>3  Symmetry  of  P  in  Proposed  HJI  Solution  V  =  x^Px 

Recall  now  that  in  Section  2.5.2  two  approaches  to  solving  the  HJI  equation  (SDRE)  associated 
with  the  nonlinear  Hqo  suboptimal  control  problem  were  proposed.  One  was  to  let  V  =  x'^Px^  with 
P  z=:  P^ ^  and  attempt  to  solve  the  Hoo  SDRE  (2.161)  and  additional  inequality  N{x)  <  0.  The 
other  was  to  let  14  =  2x^P^  with  P  not  necessarily  symmetric,  solve  the  SDRE,  and  attempt  to 
solve  the  PDE  above  to  obtain  a  (locally)  positive  definite  V.  We  now  consider  a  third  alternative, 
motivated  by  the  successful  application  of  Method  2  with  nonsymmetric  P  to  the  example  problem  of 
Chapter  3.  This  alternative  is  to  consider  a  solution  of  the  form  V  =  x'^Px^  with  P  not  necessarily 
symmetric,  and  investigate  the  utility  of  removing  the  symmetry  assumption  on  P.  It  turns  out 
that  no  utility  is  added  by  removing  the  symmetry  assumption  on  P  since  only  the  symmetric  part 
contributes  to  the  solution,  as  shown  below. 

Recall  that  we  wish  to  solve  the  HJI 

H*  =VJ  +  z'^z  -  j^d'^d  =  V^  Ax-  +  ^GG'^Vj  +  x'^H'^Hx  <  0  (4.24) 

With  V  ~  x^Px  and  using  (4.14)  we  have 

SO  that  using  (2.166)  we  see  that  (4.25)  can  be  written 

14  =  x'^[P  +  P^  +  Mp]  (4.26) 

Now,  if  we  write  P  as  the  sum  of  its  symmetric  and  skew-symmetric  parts  [42] 

P  =  Psym  +  Psk  (4.27) 

where 

Psym  =  \{P  +  P'^)  (4.28) 
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and 


Psk  =  ^{P  -  P'^) 

we  immediately  see  from  (4.28)  that  P  +  =  2Psym,  so  that  (4.26)  becomes 


(4.29) 


Vx  =  x'^[2Psym,  +  Mp] 


(4.30) 


Substituting  (4.30)  into  (4.24)  and  simplifying  we  obtain 


x'^  [A^Paym  +  PsymA.  +  PsymKPsym  +  H\  X  +  : 


Mp{A  +  KPsym  +  ^KM^) 


a;<0  (4.31) 


where  we  have  defined 


K  =  ■\gG'^  -  BB'^ 


(4.32) 


Thus,  we  can  see  that  the  SDRE  part  of  (4.31)  involves  only  the  symmetric  part  of  P,  with  the 
skew-symmetric  part  contributing  nothing.  Now  we  must  consider  the  second  part  of  (4.31),  which 
contains  the  term  Mp.  If  we  decompose  P  according  to  (4.27)  in  (2.166),  we  can  write  Mp  as 
Mp  =  Ml  -\-  M2  where 

(4.33) 

(4.34) 


OX 


dPsk 

M2  =  — — X 


dx 

Now,  using  the  fact  that  for  any  skew-symmetric  matrix  Pjf,  ~  -Pski  so  that  x'^PgkX  —  0  for  all  x, 
we  have 


dx'^Fs 


dx 


kX  X 

—  =  x 


Pjk+P^k+ 


=  0 


so  that 


Thus,  (4.31)  reduces  to 


c'^^^x  =  x'^M2=0 
ox 


(4.35) 


(4.36) 


T 


Psym  "b  PsymA  A  P P sym  ~b  P  -^]  ^  A  i 


Mx{A  +  KPy^  A  -^KMl) 


x<0  (4.37) 


so  that  indeed  the  solution  to  (4.24)  depends  only  on  the  symmetric  part  of  P. 
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Simplification  of  SDRE  Necessary  Condition  for  Optimality 

Recall  that  in  the  SDRE  nonlinear  regulator  problem  we  wish  to  find  the  control  u  which  solves  the 
following  optimal  control  problem  equation  (note  the  x  dependency  notation  has  been  dropped  for 
brevity): 

1  1 

0  =  min  H  =  min  -x^Qx  4-  -u^Ru  +  X^[Ax  +  Bu]  (4.38) 

u  u  2  2 

By  invoking  the  first-order  necessary  condition  Hu  =  0^  and  assuming  R  —  jf?^,  we  obtain 

u'fR-\-X^B  =  0  (4.39) 

where  we  have  used  the  notation  Uo  for  the  optimal  control  Making  the  association  A  —  Px  and 
solving  we  get 

Uo{x)  -  -R-^B'^Px  (4.40) 

Substituting  (4.40)  into  (4.38)  and  simplifying  leads  to 

x'^lA^P  H-  P'^A  -  P'^BR-^B'^P  AQ]x  =  0  (4.41) 

which  can  be  solved  by  setting  the  term  in  brackets  equal  to  zero,  yielding  the  state-dependent 
Riccati  equation  in  the  case  where  we  assume  P  —  P^  as  in  Section  2.5.1.  In  [13]  the  authors  give 
an  additional  vector  equation  of  dimension  n  which  must  be  satisfied  in  order  for  the  SDRE  method 
to  satisfy  the  necessary  conditions  of  the  optimal  nonlinear  regulator  problem.  This  equation,  called 
the  SDRE  Necessary  Condition  for  Optimality,  is  related  to  requiring  the  solution  to  follow  the 
optimal  costate  vector  trajectory.  The  equation  given  in  [13]  and  repeated  in  Section  2.5.1  is  (again 
assuming  P  =  P^) 

Pa;  +  -x'^&x]  +  lx'^PBR-^[^x]R-^B^Px  +  x'^[^x]Px  -  x'^PBR-^[^x]Px  =  0  (4.42) 
2  dx  2  ox  ox  ox 

which  is  a  complicated  equation  involving  third-order  tensors  and  the  time  derivative  of  the  P 

matrix.  If  we  were  to  factor  out  an  x  to  the  right  in  (4.42)  and  then  set  the  premultiplying  term 

equal  to  zero,  we  would  have  constraint  equations  to  solve.  In  this  section  we  derive  an  alternative 

form  for  (4.42)  which  is  significantly  simpler,  also  providing  a  physical  explanation  for  the  resulting 
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requirement  and  showing  that  for  symmetric  P  the  necessary  condition  for  optimality  actually  places 
n{n  —  l)/2  as  opposed  to  additional  constraints  on  the  control  problem.  The  derivation  of  this 
simplified  necessary  condition  takes  advantage  of  the  fact  that  (4.41)  is  satisfied  by  our  choice  of  A 
and  P,  and  is  presented  below. 

We  start  by  considering  the  most  general  case  in  which  P  is  not  necessarily  symmetric.  Differ¬ 
entiating  A  =  Px  ,  for  the  closed  loop  system  we  obtain 


<  dPx  . 
A“  = 
ox 


dx 


X  P 


{Ax  +  Buo) 


(4.43) 


Transposing,  we  get 


=  {x^A^  +  u^B^) 


P^  +  x 


dx 


For  optimality,  we  need  to  satisfy 


\T  — 

dx 


(4.44) 


(4.45) 


Evaluating  the  right  hand  side  of  (4.45)  (recalling  that  u  and  A  are  considered  variables  independent 
of  X  for  purposes  of  this  operation)  we  find 


dA  ,  dB 

—X  +  A  +  —U 
OX  ox 


(4.46) 


Using  the  fact  that  on  the  optimal  trajectory  rt  —  we  have 


\T  T  ^  1  ^  jrdR  ,  j, 

A  =-x  Q--X 


dA  ,dB 

-^x  +  i4  + 
ox  ox 


(4.47) 


Now,  recall  that  in  satisfying  (4.41)  we  are  actually  just  satisfying 


1  1 

li{x,  Uo)  =  -x^Qx  +  -u^Ruo  +  x^ P'^[Ax  T  BU(^  —  0 


(4.48) 


for  all  X.  Thus,  taking  the  partial  derivative  of  (4.48)  with  respect  to  x  we  get 
x^Q  +  +  \u'^[^Uo]  +  [ujR  +  x'^P'^B]^ 


T  oT 


-^X^P 


dA  .  OB 

-—X  -h  -h  -^Uo 

ox  ox 


+  {x'^A^  +  u^B'^) 


P  +  -^x 
ox 


=  0 


(4.49) 


Replacing  x^P^  with  A^,  using  (4.39)  so  that  the  coefficient  of  ^  vanishes,  and  rearranging  gives 


T/O  ^  1  ^  ^rdR  , 


dA  ,dB 

—x+A  +  —Wo 

OX  ox 


=  (x^A'^  +  ujB^) 


o  9P 
P  +  —X 
dx 


(4.50) 
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Substituting  (4.50)  into  (4.47),  we  find 


jr  nx  j_,,TrT', 

Lq  . 


r.  dP  ' 

P+—X 

ox 


(4.51) 


Thus,  comparing  (4.51)  to  (4.44),  we  see  that  to  achieve  the  optimal  solution  we  must  have 


(x^A^  +  ujB'^) 


P'^  +  x'^- 


dx 


R  9P 
Pp-^x 

OX 


(4.52) 


This  is  the  simplified  necessary  condition  for  optimality  which  is  significantly  easier  to  work  with 
than  (4.42),  for  clearly  a  sufficient  condition  for  (4.52)  to  hold  is  to  have 


P^  +x 


OX 


dx ' 


(4.53) 


If  we  now  enforce  the  assumption  that  P  =  as  in  [13],  the  SDRE  simplifies  to 


A^  P  ~hFA~  PBR-^B'^P  +  (5  =  0 


while  (4.53)  becomes 


or 


rdP  dP 
^  dx  ~  dx^ 


tOP  dP  . 
- - —X  =  0 


(4.54) 


(4.55) 


(4.56) 


dx  dx^ 

An  interpretation  of  (4.53)  is  provided  by  making  use  of  an  insight  from  Hamilton- Jacobi- Bellman 
theory.  In  HJB  theory,  the  optimal  costate  differential  equation  is  automatically  satisfied  when  the 
HJB  equation  is  solved  (see  [8]).  The  key  reason  is  that  =  14,  where  V  is  the  optimal  cost  for  the 
control  problem.  Thus,  the  costate  vector  is  a  perfect  differential,  so  that  its  partial  derivative  with 
respect  to  x  yields  the  Hessian  matrix  of  F,  which  is  symmetric.  This  symmetry  is  key  to  cancellation 
of  all  terms  in  the  proof  of  satisfaction  of  =  —Hx  iii  [8]*  This  suggests  (4.53)  is  equivalent  to  the 
requirement  that  the  costate  vector  be  a  perfect  differential,  so  that  the  optimal  cost  V  exists  and  is 
well-defined.  This  conjecture  is  validated  when  one  sets  -1^  =  -^^5  corresponding  to  the  symmetric 
Hessian  requirement  for  V.  In  so  doing  one  obtains  (4.53).  Thus,  satisfaction  of  (4.53)  is  in  fact 
equivalent  to  existence  of  an  optimal  cost  F,  for  which  =  x^P^  is  its  differential  (there  is  an  if 
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and  only  if  correspondence).  This  equivalence  is  stated  in  [45],  and,  in  addition,  a  formula  for  an 
appropriate  V  (appropriate  in  the  sense  that  V  (0)  =  0)  is  given  as 

V{x)  =  2x^  r  P{tx){tx)dt  (4.57) 

Jo 

Also,  it  is  stated  that  if  F  is  a  positive  definite  matrix-valued  function,  then  V  is  positive  definite. 
Thus,  (4.52)  is  required  to  be  satisfied  in  order  to  achieve  optimality,  but  satisfaction  of  (4.53)  is 
at  least  sufficient  (and  perhaps  necessary)  for  satisfying  (4.52),  and  necessary  in  order  to  have  a 
well  defined  (C^)  optimal  cost  function  exist.  We  additionally  would  like  to  have  a  positive  definite 
solution  P  so  that  V  is  positive  definite,  enabling  its  use  as  a  Lyapunov  function  for  the  system. 

We  see  that  in  general  (4.53)  is  a  matrix  equation  in  x  which  imposes  scalar  constraints 
on  the  optimization  problem.  For  P  ~  P'^  however,  by  examining  the  expanded  version  of  (4.56) 
using  (4.7)  and  (4.13),  it  can  be  seen  that  the  diagonal  elements  of  the  left  hand  side  of  (4.56)  are 
automatically  zero  due  to  the  symmetry  assumption  on  P.  Also,  the  off-diagonal  elements  of  the 
left  hand  side  of  (4.56)  are  equal  to  their  symmetric  counterparts  due  to  the  symmetry  assumption. 
Thus,  in  the  case  of  symmetric  P,  the  left  hand  side  of  (4.56)  is  a  symmetric  matrix  function  so  that 
(4.56)  imposes  k  =  n{n  —  l)/2  constraints  on  the  problem. 

4^5  Relationships  Between  Solution  Methods  for  SDRE  Nonlinear  Hqq  Control 

In  this  section  we  build  on  the  insights  of  Sections  4.3  and  4.4  to  develop  relationships  between 
Methods  1  and  2  proposed  in  Section  2.5.2  for  solving  nonlinear  Poo  control  problems  via  the  SDRE 
technique.  We  assume  the  existence  of  stabilizable  and  detectable  factorizations.  Suppose  we  solve 
the  Hoo  SDRE 

A^P  +  PA  +  PKP  +  H'^H  =  0  (4.58) 

where  K  is  as  defined  in  (4.32),  and  P  is  assumed  symmetric.  Then  suppose  we  seek  to  use  Method 
2  of  Section  2.5.2  so  that  we  have  Vx  =  2a:^P,  and  from  Section  4.4  we  know  that  a  solution  V  to 
the  PDE  above  only  exists  if  a?^P  is  a  perfect  difterential  of  a  scalar  function  of  x,  or  equivalently  if 

Mp  =  MJ  (4.59) 
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where  Mp  is  defined  as  in  (2.166).  If  we  suppose  (4.59)  holds,  then  V  is  given  as  in  Section  4.4  by 


V{x)  =  2^ 


Tpit 

Jo 


x){tx)dt 


Now  let  us  compare  these  results  to  the  results  of  using  Method  1  of  Section  2.5.2.  Using  Method  1, 
suppose  we  also  solve  (4.58)  and  satisfy  (4.59),  and  additionally  our  solution  P  is  such  that  we  have 

N{x)  =  MpA  +  ^MpKMp  +  MpKP  =  0  (4.61) 

It  is  easily  shown  that  simultaneously  satisfying  (4.58),  (4.59),  and  (4.61)  is  the  same  as  satisfying 
the  algebraic  Riccati  equation 


A^Pi  +  PiA  +  PiKPi  -f  -  0 


where 


(4.63) 


However,  the  two  Riccati  equations  (4.58)  and  (4.62)  are  the  same,  so  that  by  uniqueness  of  solutions 
we  must  have  Pi  ==  P,  and  thus  by  (4.63)  we  must  have  Mp  “  0.  This  is  seen  alternatively  by 
rearranging  (4.61)  to  the  form  of  an  algebraic  Riccati  equation 


Mp{A  -f  KP)  +  (^  +  KPfMp  +  Mp{-K)Mp  =  0 


Now,  notice  that  A-\-  K  Pis  guaranteed  Hurwitz  by  our  assumptions,  and  there  is  no  state  penalty 
matrix  in  (4.64).  It  is  well  known  that  such  a  Riccati  equation  has  the  unique  stabilizing  solution 
Mp  —  0.  Thus,  what  we  have  shown  is  that  if  P  :=  P^  and  Mp  =  Mp^  then  if  P  is  a  solution  to 
the  SDRE,  Method  2  has  a  solution  (for  iV  =  0)  if  and  only  if  Mp  =  0,  and  the  solutions  from  the 
two  methods  are  identical.  What  this  in  turn  implies  is  that  P  is  a  constant  matrix,  so  that  we  see 
that  no  matter  which  method  we  use  to  compute  V  we  get 


V  =  x^Px  —  2x^P  /  txdt 


In  summary,  we  have  the  following  results.  Assume  P  is  a  solution  to  the  SDRE  (4.58),  so  that 


P^P^.  Then 
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•  if  Mp  =  Mp  5  Method  2  is  solvable  and  V  is  given  by  (4.60).  Method  1  is  solvable  (with  iV  =  0) 
iff  ATp  =  0,  so  that  P  is  a  constant  matrix,  and  both  methods  give  the  same  answer 

•  if  Mp  ^  Mp,  then  neither  method  is  solvable  for  V 

What  this  basically  means  is  that  Method  1  offers  no  advantages  over  Method  2  except  possibly  in 
the  case  where  we  allow  iV  <  0,  so  that  Method  2  is  the  preferred  solution  method. 

To  close  this  section,  we  observe  that  the  above  analysis  depends  on  P  being  symmetric  in 
Method  2,  but  we  have  seen  in  Chapter  3  that  both  methods  can  yield  valid  local  solutions  if  P  is 
allowed  nonsymmetric.  Section  4.3  proves  that  P  can  never  be  nonsymmetric  in  Method  1.  Allow¬ 
ing  nonsymmetric  P  in  Method  2,  however,  eliminates  the  possibility  of  using  standard  numerical 
Riccati  equation  solvers,  and  thus  mandates  the  analytical  solution  of  the  Riccati  equation  and  PDE 
involving  Mp.  Research  performed  by  this  author  indicates  that  analytical  solutions  to  SDREs  will 
be  diflS.cult  to  obtain  in  most  problems  of  interest,  and  thus  we  shall  consider  P  =  P^  and  focus  on 
numerical  Riccati  solutions  in  the  remainder  of  this  dissertation. 

4^  Optimal  Control  Numerical  Solution  Algorithm 

As  mentioned  in  the  previous  section,  in  the  course  of  this  research  several  failed  attempts  at 
analytical  solutions  to  simple  multistate  example  problems  (n  =  2)  were  made,  and  we  thus  decided 
to  focus  on  numerical  solutions.  We  therefore  developed  and  implemented  a  numerical  SDRE  solution 
approach  to  the  optimal  nonlinear  regulator  problem  based  on  the  results  of  Section  4.4.  In  [13],  the 
authors  suggest  two  potential  ways  of  solving  the  SDRE  necessary  condition  for  optimality  (4.42). 
One  is  to  assume  the  SDC  factorization  parameter  vector  a  is  an  explicit  function  of  time,  and  the 
other  is  to  assume  a  is  an  explicit  function  of  the  state  vector  x.  They  then  outline  an  algorithm  to 
solve  (4.42)  based  on  n  =  a{t)^  which  involves: 

Procedure  1 

i.  choosing  an  q:(0) 
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ii.  solving  the  SDRE  for  P{x,  a) 


iii.  differentiating  the  SDRE  with  respect  to  the  Xi  and  (Xi  to  obtain  Lyapunov  equations  for  the 
Px.  and  Pai 

iv.  solving  those  Lyapunov  equations 

V.  substituting  back  into  the  necessary  condition  for  optimality  and  algebraically  solving  to  obtain 
a 

vi.  integrating  over  one  time  step  and  returning  to  step  ii  above  until  a(t)  diverges 

vii.  iteratively  adjusting  a(0)  until  d(i)  — >  0  as  t  ^  oo. 

This  is  quite  a  cumbersome  procedure  having  no  hope  of  real  time  implementation  since  in  the  last 
step  one  must  somehow  compute  how  slight  changes  in  a(0)  affect  things  farther  and  farther  out  in 
time,  and  change  a(0)  accordingly.  We  propose  instead  a  solution  procedure  similar  to  the  above 
but  based  on  combining  both  assumptions;  that  is,  a  —  a{x,  t).  At  each  time  step  we  let  each  ai 
be  expressed  as  a  polynomial  (of  a  fixed,  user-desired  order)  in  x  with  unknown  constant  (over  one 
time  step)  coeflScients  an.  The  proposed  method  is  then  as  follows: 

Procedure  2 

i.  stack  the  an  into  a  new  unknown  vector  6 

ii.  choose  (^o  =  [0  ...  0]^  may  be  a  good  choice) 

iii.  solve  the  SDRE  for  P{x^d) 

iv.  follow  step  iii  in  Procedure  1,  substituting  Oi  for  ai 

V.  differentiate  the  expressions  from  step  iv  above  with  respect  to  Xi  and  Oi  to  obtain  Lyapunov 
equations  for  PxiOi  and  Pq2  (the  partial  of  P  with  respect  to  Oi  twice) 

vi.  use  the  information  from  the  previous  step  to  form  the  gradient  of  our  simplified  necessary 
condition  for  optimality  (4.56)  with  respect  to  the  unknown  parameter  vector  9 
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vii.  use  an  iterative  root  finding  procedure  such  as  Newton’s  method  to  calculate  the  next  value  of 
6  (note:  we  may  not  need  the  previous  step  if  we  choose  another  method  that  does  not  require 
gradients) 

viii.  return  to  step  iii  and  iterate  until  6  converges 

ix.  use  the  obtained  values  of  P  and  a  to  form  the  closed  loop  system 

X.  integrate  to  the  next  time  step  and  repeat,  using  the  last  value  of  6  as  the  new  initial  guess  in 
step  ii 

Note  that  by  using  this  procedure  we  have  converted  our  optimal  control  problem  into  a  parameter 
optimization  problem  to  be  solved  at  each  time  step.  Also  note  that  this  procedure  actually  allows 
a  to  be  a  function  of  both  time  and  the  state  vector,  since  it  is  possible  a  time- varying  6  vector 
may  be  obtained.  Such  a  methodology  has  at  least  two  advantages  over  Procedure  1.  First,  we  have 
a  good  initial  guess  for  since  any  value  of  6  with  only  zeros  or  ones  in  the  constant  coefficient 
parts  comprises  a  valid  parametrization  which,  if  held  constant  over  the  duration  of  the  simulation, 
will  from  experience  yield  sub  optimal  performance  close  to  optimal.  Second,  we  do  not  need  to 
integrate  the  dynamics  into  the  future  to  adjust  6  iteratively.  This  represents  a  major  reduction  in 
computational  burden  and  a  great  simplification  in  implementation. 

This  procedure  was  implemented  in  Matlab/Simulink  for  Example  2  of  [14],  the  parameters  of 
which  are  given  below: 

—  Xi  -  x\-\- X2 Ui 

X2  =  ^1  +  -  X2'^U2  (4.66) 

with  H  —  I2  and  R  —  212^  where  I2  is  the  two  dimensional  identity  matrix.  This  is  a  second- 
order  example,  so  that  n  is  a  scalar  (function  of  x).  Using  only  a  constant  term  in  the  polynomial 
expansion  for  a,  we  were  able  to  duplicate  the  optimal  and  suboptimal  control  and  state  trajectories 
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given  in  [14],  starting  from  the  initial  condition  xq  —  [1  1]^ 


where  for  the  suboptimal  case  we  have 


A{x)  =  A2{x)  = 


1  —  x\  1 
l-\-XiX2  -1 


(4.67) 


These  histories  are  given  in  Figures  4.1,  4.2,  and  4.3,  where  the  suboptimal  trajectories  have  been 
specifically  labeled  with  an  ^so’  suffix,  and  the  neighboring  optimal  trajectories  have  been  left  un¬ 
labeled.  Interestingly  enough  we  did  not  obtain  the  same  nonconstant  a  time  history  as  Cloutier 
et  al  This  seems  to  suggest  possible  nonuniqueness  of  a.  Initial  condition  tradeoff  studies  were 
also  performed.  With  ao  fixed,  the  simulation  was  repeated  for  various  initial  x  conditions,  in  effect 
mapping  out  a  small,  elliptical  shaped  region  of  convergence  centered  at  the  origin  of  the  state  space 
for  this  algorithm.  With  xq  confined  to  this  region,  convergence  was  obtained  for  various  values 
of  ao  G  [—1.5,  1.5],  and  nonconvergence  for  ao  outside  this  region  (both  plus  and  minus).  In  the 
convergent  cases  very  few  intermediate  Newton  iterations  were  required.  Another  interesting  phe¬ 
nomenon  observed  was  that,  by  choosing  small  magnitude  initial  conditions,  we  did  indeed  obtain  a 
constant  (with  respect  to  time)  profile  of  a,  although  the  values  obtained  differed  depending  on  the 
initial  conditions  chosen.  This  appears  to  support  the  idea  of  strong  x  dependency  in  a  and  using  a 
Taylor  series  expansion  in  x  to  represent  a,  since  constant  values  of  a  were  observed  for  very  small 
X  values.  Finally,  we  explored  the  use  of  this  method  with  the  a  updating  turned  off.  For  all  values 
of  a  chosen  (ranging  from  -10,000  to  10,000),  stable  trajectories  which  displayed  the  same  type  of 
behavior  in  not  satisfying  the  necessary  condition  for  optimality  (4.56)  were  obtained.  Virtually  all 
profiles  were  smooth  as  in  Figure  4.4,  somewhat  large  in  the  beginning  and  converging  rapidly  to 
zero  within  a  couple  of  seconds  (the  initial  condition  responses  decayed  to  zero  within  about  four 
seconds  in  all  cases). 

Although  this  algorithm  proved  successful  in  obtaining  the  optimal  control  within  a  small  radius 
of  convergence  from  the  origin,  simple  attempts  to  extend  the  convergence  radius  failed.  As  the 
suboptimal  trajectories  differed  only  slightly  from  the  optimal  ones,  the  payoff  in  seeking  optimal 
controls  at  the  cost  of  a  high  increased  workload  was  judged  to  be  insufficient  to  warrant  further  effort 
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in  this  area.  Suboptimal  control  strategies  based  on  SDRE  solutions  for  fixed  SDC  parametrizations 
^vere  instead  pursued. 

4^  7  Solvability  oj  HJIs  and  HJB  Equations 

In  this  section  we  give  some  brief  qualitative  comments  regarding  the  solvability  of  HJIs  and  HJB 
equations  in  general,  and  in  particular  their  solution  by  SDRE  methods,  where  by  solvable  we  mean 
that  a  satisfactory  positive  semidefinite  storage  (cost)  function  may  be  found  which  solves  (2.125)  or 
(2.135).  We  first  give  a  summary  of  known  results,  and  then  proceed  with  some  original  observations. 

Solution  of  HJIs  is  in  general  nontrivial  and  relatively  few  proposed  solution  techniques  or  solved 
problems  can  be  found  in  the  literature.  In  1969,  Lukes  [47]  presented  an  inductive  power  series 
approximate  solution  method  for  Hamilton- Jacobi-Bellman  equations  like  (2.135).  Van  der  Schaft 
extended  this  method  to  Hamilton- Jacobi- Issacs  equations  in  [66].  A  similar  approximate  power 
series  solution  method  is  proposed  in  [29].  In  [71]  and  [72]  an  iterative  power  series  approach 
was  used  to  solve  the  state  feedback  Hamilton- Jacobi-Issacs  equation  approximately,  where  the 
quadrat ric  term  in  V{x)  and  thus  the  linear  part  of  the  control  was  obtained  via  a  gain-scheduled 
solution  to  the  linearized  problem.  In  [48],  the  method  of  characteristics  was  used  to  solve  an  iJoo 
aerospace  plane  ascent  problem  numerically.  Analytical  nonlinear  Hoo  solutions  have  been  shown 
to  exist  for  state  feedback  control  of  rigid  spacecraft  in  [37]  and  [16],  and  an  analytical  output 
feedback  solution  was  given  for  the  very  special  case  of  passive  systems  with  an  assumed,  restrictive 
structure  in  [15].  An  output  feedback  solution  for  lossless  systems  of  special  structure  was  also 
given  in  [67].  In  [45]  a  solution  approach  based  on  nonlinear  matrix  inequalities  (NLMIs),  which 
are  actually  state- depen  dent  LMIs,  is  proposed.  This  approach  is  actually  a  generalization  of  the 
SDRE  theory  considered  in  this  dissertation.  This  becomes  clear  when  one  examines  [30],  in  which 
the  NLMI  approach  is  implemented  via  a  finite  difference  scheme,  and  a  single  common,  constant, 
local  solution  to  multiple  point  wise  LMIs  is  sought.  From  this  discussion  it  can  be  seen  that  the 
number  of  solved  nonlinear  Hoo  control  problems  is  very  small,  and  virtually  every  proposed  method, 
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except  the  recent  NLMI  method,  seeks  solutions  to  the  Hamilton-Jacobi-Issacs  equation  (HJIE),  as 
opposed  to  the  strict  inequality  form  of  the  HJL  This  observation  motivates  a  brief  discussion  of  the 
difference  between  solving  strict  HJIs 

n{Va,,x,u^,d^)  (4.68) 

versus  solving  the  equality  form  (HJIE) 

(4.69) 

As  mentioned  in  the  above  discussion,  most  solution  attempts  in  the  literature  focus  on  (4.69). 
However,  in  [32],  Imura  et  al  develop  theory  for  the  strict  inequality  case  based  on  a  strict  bounded 
real  condition  for  nonlinear  systems.  This  is  done  to  build  internal  stability  directly  into  the  problem, 
by  requiring  a  positive  definite  HJI  solution  with  strictly  negative  definite  derivative  (when  d  =  0), 
which  serves  as  a  Lyapunov  function  to  guarantee  asymptotic  stability  of  the  closed  loop  system. 
The  advantage  gained  from  solving  (4.68)  (thus  requiring  negative  definite  V)  is  seen  to  be  from 
Theorem  2.1.4  that  all  bounded  closed  loop  solutions  are  guaranteed  to  converge  to  x  =  0.  Thus, 
boundedness  implies  closed  loop  asymptotic  stability  if  (4.68)  holds.  Of  course,  when  F  >  0  as  in 
[32],  Lyapunov’s  Theorem  may  directly  be  used  instead  of  Lasalle’s  Invariance  Principle  to  guarantee 
asymptotic  stability.  For  the  equality  case,  (4.69)  just  guarantees  F  <  0  when  d  =  0,  Thus,  this 
approach  requires  the  use  of  Theorem  2.1.4  and  its  additional  complexities  to  guarantee  closed  loop 
stability.  It  is  easily  seen  from  (2.126)  that  a  negative  definite  V  can  be  recovered  from  (4.69)  by 
simply  requiring 

h{x)  =0  <:^x^0  (4.70) 

so  that,  when  (4.70)  holds,  there  is  no  advantage  to  solving  the  strict  HJI  (4.68).  On  the  other  hand, 
a  strict  HJI  (4.68)  can  easily  be  transformed  into  the  form  of  (4.69),  if  we  modify  the  output  h  to 
be  globally  nonsingular  as  in  (4.70).  To  illustrate,  suppose  we  wish  to  solve  (4.68).  This  may  be 
accomplished  by  setting 

n{Va:,X,U^,d^)<-w{x)  (4.71) 
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\\rhere  w(x)  is  a  positive  definite  function  of  x.  Thus,  -~w{x)  <  0  V  x  ^  0,  ^f;(0)  =  0,  so  that  (4.71) 
and  in  turn  (4.68)  are  satisfied  if  we  solve  the  HJIE 

n{V:,,x,u,,d^)-]-w{x)  =  0  (4.72) 

In  terms  of  the  SDRE  method,  if  we  have  a  fixed  h  and  we  desire  to  solve  the  strict  inequality,  we 
can  instead  solve  the  HJIE 

A^P  +  +  PBKB^P  +  Q  =  0  (4.73) 

where  we  have  the  new  state  weighting  matrix  function  Q  ~  H'^H  +  IT  >  0,  where  w  ~  x^Wx  is 
any  globally  positive  definite  function,  i.e.,  W  —  el^  0  <  e  «  1.  Thus,  solving  (4.68)  is  essentially 
equivalent  to  solving  an  HJIE,  where  h'^h  is  restricted  to  be  positive  definite.  In  the  SDRE  context 
this  requires  Q  =  >  0  V  a?,  which  in  turn  implies  P{x)  >  0\/  x.  In  solving  (4.69),  however,  we 

can  allow  h'^h  to  be  only  positive  semidefinite,  thus  opening  up  design  options  not  available  if  we 
solve  (4.68).  Closed  loop  stability  may  be  more  complicated  to  guarantee,  but  the  additional  design 
flexibility  may  prove  extremely  useful.  These  issues  are  explored  further  in  Chapters  8,  9,  and  12. 

We  now  move  on  to  discuss  issues  common  to  solving  HJIs,  regardless  of  whether  we  choose 
to  solve  the  strict  inequality  or  the  equality  form.  We  consider  both  system  type  properties  and 
algorithm  type  properties  specific  to  the  SDRE  methods.  In  the  first  category  we  give  a  necessary 
condition  for  solvability  of  an  HJI/HJB  equation  by  any  control  strategy  (which  has  to  do  with 
nonlinear  controllability  issues),  and  in  the  second  category  we  are  concerned  with  the  effects  of 
various  choices  of  SDC  parametrizations  in  the  SDRE  solution  approaches.  Letting  d=0,  we  first 
consider  the  nonlinear  regulator  type  problem.  As  per  [33],  a  system 

X  —  a{x)  +  b{x)u 

may  locally  be  decomposed  about  any  point  of  interest  into  state  vector  dynamics  which  are 
affected  by  the  control,  and  state  vector  dynamics  ±2  which  are  unaffected  by  the  control,  with 
dimensions  depending  on  the  dimension  dc  of  the  smallest  nonsingular  involutive  distribution  Ac 
invariant  under  a  and  the  hi  and  also  containing  the  span  of  the  hi  (dim(a^i)  =  dc^  dim(a?2)  ~n~dc^ 
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where  n  is  the  dimension  of  the  state  space).  The  decomposed  system  dynamics  take  the  form 


±1  =  ai{xi,X2)  +  bi(xi,X2)u 

X2  =  a2{x2)  (4.74) 

Thus,  if  dc  <  n,  the  X2  states  will  evolve  according  to  a2  along  some  input-independent  manifold  in 
the  state  space.  If  the  system  does  not  converge  to  the  origin  along  this  manifold,  then  the  overall 
system  cannot  be  stabilized  to  the  origin,  regardless  of  what  control  is  selected.  Since  finding  a  locally 
positive  definite  solution  to  the  HJB  equation  for  a  zero  state  observable  system  implies  a  closed 
loop  system  which  is  stabilized  to  the  origin,  we  obviously  will  not  be  able  to  find  such  a  solution 
if  such  an  unstable  input- independent  manifold  exists.  A  necessary  condition  for  local  solvability  of 
the  nonlinear  regulator  SDRE  is  therefore  that  no  such  unstable  input-independent  manifold  exists, 
or  equivalently,  that  the  system  be  nonlinear ly  stabilizable.  The  theory  may  be  extended  [33] 
to  the  global  case  by  considering  the  span  of  the  Control  Lie  Algebra  instead  of  the  distribution  Ac 
described  above,  an  extension  which  also  allows  for  singular  distributions.  Thus,  prior  to  attempting 
to  solve  a  given  problem,  we  should  consider  the  Control  Lie  Algebra  for  the  problem,  and  determine 
if  and  where  such  uncontrollable,  unstable  manifolds  exist.  Note  that  this  applies  to  nonlinear  ffoo 
as  well  as  nonlinear  regulation,  since  both  require  stabilization  to  the  origin  in  the  closed  loop. 
These  issues  are  explored  in  greater  detail  in  Chapter  6.  In  addition  to  the  system  controllability 
properties,  we  require  various  assumptions  on  the  SDC  factorizations  to  guarantee  solvability  of 
HJI/HJB  equations  via  the  SDRE  techniques.  In  the  case  of  analytic  systems  describable  by  a 
single  state  [n  —  1),  sufficient  conditions  for  solvability  are  given  in  Chapter  5,  as  well  as  necessary 
conditions  for  most  cases  of  interest.  In  the  multistate  case  we  are  motivated  by  the  results  of  Sections 
4.5  and  4.6  to  forego  solving  the  HJI/HJB  equations  to  obtain  the  optimal  solution  and  associated 
cost  function  to  use  as  a  Lyapunov  function,  but  instead  to  consider  suboptimal  solutions  based  on 
SDREs  alone,  seeking  alternate  forms  of  Lyapunov  functions.  The  effects  of  SDC  parameterizations 
on  this  process  are  explored  in  detail  in  Chapters  6,  7,  8,  9,  and  10. 
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V,  Solution  Properties  for  Scalar  Analytic  Systems 


5J  Introduction 

In  this  chapter  we  examine  the  SDRE  nonlinear  regulator  for  the  scalar  (single-state)  analytic  case, 
giving  necessary  and  sufficient  conditions  for  obtaining  (locally)  stabilizing  solutions  for  almost  all 
possible  sets  of  assumptions  on  dynamics,  input  matrices,  and  weighting  functions.  Although  obvi¬ 
ously  not  widely  applicable,  these  results  offer  significant  insight  into  the  nature  of  SDRE  solutions 
and  their  relationships  with  the  linear  theory. 

As  in  Section  2.5.1,  we  consider  regulation  of  input-affine  nonlinear  dynamical  systems,  but 
here  we  limit  ourselves  to  systems  describable  by  a  single  state  variable,  x.  We  assume  perfect 
measurements  and  that  the  system  has  an  equilibrium  at  the  origin.  For  such  a  system  we  may 
write 


X  =  a{x)  H-  h{x)u^  a(0)  =  0 
h{x) 

'  /i(0)  -  0 


(5.1) 


L  "  J 

where  ti  is  a  scalar  control,  js:  is  a  scalar  penalized  variable,  and  a,  h  and  h  are  assumed  analytic 
real- valued  functions  of  x.  The  control  objective  is  minimization  of  the  cost  function 


1 

J=^  h^-\-  v?dt  (5.2) 

2  JQ 

In  [13]  it  is  shown  that  the  above  objective  may  be  accomplished  by  using  the  following  nonlinear 
regulator  SDRE  technique: 


i.  Write  (5.1)  in  the  so-called  state-dependent  coefficient  (SDC)  form 


X  =  A{x)x  +  B{x)u 
H{x)x 


(5.3) 
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ii.  Solve  the  SDRE 


A{x)p{x)  H-  p{x)A{x)  -  p{x)B^{x)p{x)  +  H^{x)  —  0  (5.4) 

iii.  Construct  the  optimal  state  feedback  via 

Uo  —  -B{x)p{x)x  (5.5) 

If  the  pairs  {^(a?),  B{x)}  and  respectively  stabilizable  and  detectable  for  all  a?, 

where  stabilizability /detectability  are  defined  in  the  standard  linear  sense,  then  it  has  been  shown 
[13]  that  the  above  algorithm  gives  a  locally  asymptotically  stable  closed  loop  system.  In  this  chapter 
we  assume  analyticity  of  the  system  parameters  a,  5,  and  h  and  seek  additional  conditions  under 
which  the  SDRE  control  algorithm  yields  an  analytic  locally  stabilizing  state  feedback.  This  line 
of  inquiry  is  motivated  by  the  fact  that,  if  the  matrices  involved  in  an  algebraic  Riccati  equation 
(ARE)  are  analytic  in  a  parameter  and  stabilizability  and  detectability  assumptions  hold,  then  the 
majcimal  solution  to  the  ARE  has  been  shown  also  to  be  analytic  in  that  parameter  [58].  Thus,  by 
assuming  analyticity  of  the  system  parameters  and  of  the  stabilizing  solution  of  (5.4),  we  expect  to 
recover  the  stabilizability /detectability  assumptions  of  [13],  at  least  as  a  special  case. 

5,2  SDRE  Solutions 

For  the  single-state  input-aflBne  case  considered  here,  the  local  solution  to  the  state  feedback  non¬ 
linear  regulator  SDRE  may  be  constructed  explicitly.  Dropping  the  x  dependency  notation  and 
rearranging,  the  SDRE  (5.4)  becomes 

B^p^  ~2Ap-  =0  (5.6) 

while  using  the  optimal  control  (5.5),  the  closed  loop  system  becomes 

X  —  Ax  -h  B{—Bpx)  =  (A  —  B^p)x  (5**^) 
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We  now  introduce  some  convenient  notation.  Recall  that  if  ^  is  a  (scalar  valued)  analytic  function 
of  X  on  i?,  then  g  has  a  unique  convergent  Taylor  series  expansion  about  the  origin 

9W  =  9(0)  +  |f  l«*  +  50lo*’+'-  (5.8) 

SO  that  near  x  —  g  acts  as  a  polynomial 

gix)  =  go  +  gix  +  +  ■  ■  ■  (5.9) 

where  the  coefficients  gi  are  simply  the  ith  derivatives  of  g  with  respect  to  evaluated  at  a;  =  0 
(weighted  by  known,  constant  scalars).  Using  (5.1)  we  may  thus  write 

a{x)  =  aix  +  a2X^  H -  (5.10) 

h{x)  =  hix  H-  h2X^  +  •  •  *  (5.11) 

so  that  A  and  H  are  given  uniquely  as 

A{x)  =  ai  +  a2X  +  -I - 

=  Aq  -f-  AiX  -h  A2X?  +  — 

H{x)  ==  +  h2X  +  hzx^  ^ - 

^E^^HrX^E2X^  (5.12) 

and  we  see  that  A  and  E  are  themselves  analytic  functions  of  x.  Now  let  dg  represent  the  minimum 
power  of  X  in  the  polynomial  expression  for  g  having  nonzero  coefficient.  Also,  let  that  nonzero 
coefficient  be  Cg.  As  per  the  above  notation,  let  ds,  dn  and  dp  represent  the  minimum  power  of 
X  with  nonzero  coefficient  in  the  polynomial  expressions  for  A,  E  and  p,  respectively.  Also,  let 
caj  chj  and  Cp  represent  the  associated  nonzero  coefficients.  We  now  give  a  well  known  stability 
lemma,  repeated  from  Section  2.1.3  for  ease  of  reference  [33]. 

Lemma  5.2.1  Consider  the  one-dimensional  system 

X  =  cx'^'^  +  Qm{x) 
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with  m  >  1,  c  ^  0,  and  Qm(^)  o.  function  vanishing  at  zero  together  with  all  partial  derivatives  of 
order  less  than  or  equal  to  m.  The  equilibrium  x  =  0  is  asymptotically  stable  if  m  is  odd  and  c  <  0. 
Otherwise  it  is  unstable. 

The  general  procedure  in  the  following  development  is  to  make  assumptions  on  the  relationships 
of  the  various  assume  existence  of  an  analytic  solution  p,  group  coefficients  of  the  smallest 
contributing  power  of  x,  and  invoke  Lemma  5.2.1  to  determine  when  stabilizing  solutions  exist.  We 
shall  consider  four  cases,  which  include  several  mutually  exclusive  subcases. 

Case  1  {dn  <  ds  and  2dH  <  dA)  For  this  case  the  lowest  order  nonzero  part  of  (5.6)  is 

=  0  (5.13) 

which,  since  is  nonzero,  has  no  solution  for  all  x.  Obviously,  there  is  thus  no  stabilizing  solution 
in  this  case.  This  result  reinforces  the  intuitive  feeling  that  we  cannot  penalize  powers  of  x  smaller 
than  those  on  which  we  may  have  some  effect  (either  through  the  control  or  through  the  dynamics 
themselves). 

Case  2  (d^  <  2^^)  With  these  assumptions  and  not  allowing  those  of  Case  1,  solutions  exist  to 
(5.6)  such  that  dp—{}  for  2dH  =  dA  and  dff  ~  ds-  Otherwise  dp  >1  and  the  existence  of  solutions 
is  highly  dependent  on  the  particular  structures  of  A,  B,  and  H,  We  thus  give  no  criteria  for  the 
existence  of  stabilizing  solutions  in  this  case.  Regardless,  examining  (5.7),  we  see  that  since  dp  >  0 
in  any  case  and  that  dA  <  2^^,  then  the  stability  of  the  closed  loop  system  is  unaffected  by  the 
control,  and  is  in  fact  determined  by  dA  and  ca-  Invoking  Lemma  5.2.1  we  conclude  for  Case  2  that 
any  solution  that  exists  is  stabilizing  iff  dA  is  even  and  ca  <  0. 

Case  3  (d^  =  2d^) 

Case  3A  {dn  >  ds)  In  this  case,  dp  =  0  and  (5.6)  reduces  to 

Cp(c|cp  -  2ca)x^^'^  -\ - =  0  (5-14) 
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Thus,  the  two  possible  solutions  to  (5.14)  are  Cp  -  0,  Cp  =  2ca/c%,  For  Cp  —  0,  the  optimal  control 
is  Uo  =  0,  giving  a  closed  loop  system  of  i  =  Ax  (to  1st  order').  Thus,  this  solution  is  stabilizing  iff 
the  open  loop  system  is  stable  {(Ia  is  even  and  <  0).  For  Cp  ~  2ca/c%,  the  closed  loop  system 
becomes  (to  leading  order) 

X  —  cax^^"^^  -h  —  ~cax^^^^  (5.15) 

Thus,  we  see  that  (5.15)  will  be  stable  iff  dA  is  even  and  ca  >  0.  Combining  these  two  results  we 
conclude  that  stabilizing  solutions  exist  for  this  subcase  iff  dA  is  even,  which  is  always  the  case  since 
dA  =  2d B^  and  dj?  is  a  nonnegative  integer. 

This  subcase  extends  to  the  situation  in  which  our  cost  function  to  be  minimized  is  identically 
zero  {djj  oo).  We  observe  that  the  nonlinear  behavior  is  analagous  to  that  observed  for  linear 
time-invariant  (LTI)  systems  in  LQR  theory.  For  LTI  systems,  if  the  open  loop  system  is  stable,  the 
LQR  optimal  control  for  zero  cost  function  is  zero,  just  as  we  see  here.  If  an  open  loop  LTI  system 
is  unstable  and  controllable,  then  the  open  loop  unstable  poles  are  moved  to  their  stable  mirror 
images  in  the  left  half  complex  plane  (i.e.,  the  real  parts  of  the  eigenvalues  of  the  closed  loop  system 
are  the  negatives  of  their  open  loop  counterparts).  In  this  nonlinear  open  loop  unstable  single-state 
case,  we  see  that  the  closed  loop  dynamics  are  the  negative  of  the  open  loop  dynamics  {ad  =  — Uoz)- 


Case  3B  {dn  ==  d^)  In  this  case  dp  again  equals  zero  and  the  leading-order  (positive)  SDRE 
solution  is  given  by 

(5.16) 


CA 


Cv  —  “?  +  W 

Substituting  (5.16)  into  Uo  =  -bpx  and  then  into  (5.1)  we  obtain  the  (leading-order)  closed  loop 
dynamics 


=  -\/4  +  (cBCi/)2a;‘^^+^ 


(5.17) 
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We  only  consider  the  positive  solution  because  it  is  trivial  to  show  that  the  negative  square  root 
solution  always  yields  an  unstable  closed  loop  system.  We  thus  see  in  this  case  that  we  get  stabilizing 
solutions  iff  (Ia  is  again  even. 


Cased  (dA  >  ^ds) 

Case  4A  (dn  =  de)  In  this  case  dp  =  0  and  (5.6)  becomes 

(5.18) 

giving 

Cp  =  \cff/cB\  (5.19) 

so  that  the  closed  loop  dynamics  become 

X  =  H -  (5.20) 

Thus,  a  stabilizing  solution  exists  iff  2^^  is  an  even  integer,  which  is  always  the  case. 

Case  4B  (djy  >  d^)  In  this  case  dp  >  1  and  the  existence  of  solutions  is  highly  dependent  on 
the  particular  structures  of  A,  B,  and  id.  We  thus  give  no  criterion  for  the  existence  of  stabilizing 
solutions  in  this  case.  We  also  note  that  such  a  case  is  somewhat  unusual,  in  that  we  would  be 
penalizing  only  large  powers  of  x  without  penalizing  the  smaller  powers. 

5,3  Discussion 

We  now  investigate  the  assumptions  required  for  local  stability  in  [13]  in  the  framework  of  the  above 
theory.  Recall  that  in  [13]  the  stability  proofs  involved  assuming  linear  stabilizability  (controllabil¬ 
ity)  of  the  pair  {A(a?),  B{x)}  V  x  and  linear  detectability  (observability)  of  the  pair  {H{x),  A(a?)}  V  x 
in  some  neighborhood  of  the  origin.  For  purposes  of  clarity  we  break  this  down  into  control¬ 
lable/observable  and  stabilizable/detectable  cases. 

For  the  single-state  controllable/observable  case  the  above  assumption  translates  to  requiring 
rank(^(a:))  =  1  and  rank(id(rr))  =  1  for  all  x  in  a  neighborhood  of  the  origin,  or,  in  effect,  B  and 
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H  both  having  nonzero  constant  parts  and  no  roots  “close”  to  the  origin.  The  controllable  case  can 
easily  be  seen  always  to  yield  stabilizing  solutions  according  to  the  above  theory  as  follows.  For 
this  case  we  have  ds  —  dff  —  0  so  that  only  cases  3B  and  iA  above  are  possible,  depending  on 
Regardless  of  the  value  of  (Ia^  it  is  trivial  to  see  that  either  case  yields  stabilizing  solutions. 

For  the  stabilizable/detectable  case,  applying  the  definition  of  linear  stabilizability/ detect  ability, 
we  must  have  ^  <  0  whenever  ^  or  H  are  equal  to  zero  {A  must  have  negative  eigenvalues  when 
not  controllable  or  observable).  Such  an  assumption  allows  dn  and  ds  to  be  greater  than  zero,  and 
also  allows  roots  ‘close’  to  the  origin,  as  long  as  the  value  of  A  at  those  locations  is  negative.  This 
interpretation  of  stabilizable/detectable  always  results  in  stable  closed  loop  systems,  as  follows.  If 
either  B  ox  H  (or  both)  loses  rank  at  a;  =  0,  then  A  must  have  a  negative  constant  part  in  order  to 
meet  the  above  definition  of  stabilizable/detectable.  Thus,  dA  —  ^  and  ca  <  0.  Only  cases  2  and  3 
are  possible,  either  of  which  yields  stabilizing  solutions.  The  only  other  possible  choice  is  for  neither 
B  nor  H  to  lose  rank  at  =  0,  but  for  both  to  lose  rank  only  at  nonzero  x  values  or  not  at  all.  In 
this  case  =  0  which  is  the  same  as  the  cont reliable /observable  case  discussed  above. 

The  results  of  this  chapter  thus  verify  the  known  result  of  analytic  and  stabilizable/detectable 
systems  yielding  analytic  stabilizing  solutions  to  (5.4).  The  results  of  this  chapter  also  show,  how¬ 
ever,  that  stabilizability/detectability  for  all  x  are  not  necessary  for  existence  of  analytic  stabilizing 
solutions  in  the  scalar  case,  and  illustrate  under  what  conditions  such  solutions  may  or  may  not  be 
obtained. 

5.4  Examples 

We  now  present  three  examples  to  illustrate  the  above  theory,  and  to  demonstrate  the  gap  between 
the  sufficient  conditions  for  stability  in  [13]  and  the  necessary  and  suflScient  conditions  for  analytic 
stabilizing  solutions  derived  herein. 

Example  1 

X  —  XU]  h  ~  cx^  (5.21) 
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This  example  is  not  controllable/observable,  nor  stabilizable/detectable  in  a  neighborhood  of  the 
origin,  so  that  the  sufficient  condition  of  [13]  tells  us  nothing.  Here  we  have  A  =  ^  B  =  x,so  that 

dA  =  2,  ca  =  1,  dB  =  1,  Cj5  —  1,  dn  —  1,  and  ch  =  c  and  Case  3B  applies.  Since  ==  2  is  even, 
we  expect  a  stable  closed  loop  solution.  Solving  (5.16)  we  find 

Cp  —  \  yjx  A  c?  (5.22) 

The  resulting  low-order  control  is 

«=-(!+  y/l+^)x^  (5.23) 

giving  the  closed  loop  system 

x  =  -\/T+^x^  +  ■■■  (5.24) 

which  is  indeed  (locally)  stable. 

Example  2 

f  -f  xu;  h  —  cx^  (5.25) 

Notice  that  all  we  have  changed  is  a{x)  from  to  x^^  so  that  again  the  sufficient  condition  of  [13] 
tells  us  nothing.  Here  we  have  A  =  x^  B  —  x,  that  dA  =  1,  ca  =  1,  ds  =  1,  =  1,  dff  —  1, 

and  Ch  ~  c  so  that  Case  2  applies.  Since  c^a  =  1  is  odd,  we  expect  an  unstable  closed  loop  solution. 
Solving  (5.6)  we  find  dp  =  1  and  Cp  —  c^/2,  so  that  p  is  not  even  locally  positive  semidefinite.  The 
resulting  low-order  control  is 

2  2 

u  =  ~x{^x)x  ~  — (5.26) 

giving  the  closed  loop  system 

x  =  x‘^  -^x^  +  ■••  (5.27) 

which  as  expected  has  stability  properties  unaffected  by  the  control,  and  is  clearly  unstable. 

Example  3 

X  ^  A  [x-  2)u]  h~  X  Ax^  (5.28) 
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In  this  example  we  have  altered  Example  2  to  have  a  controllable/observable  linearization,  which 
is  known  [67]  to  guarantee  a  locally  stabilizing  solution.  We  now  have  A  =  x,  B  =  x  —  2,  so  that 
dA  =  1,  ca  =  1,  dB  -  0,  cb  ~  -2,  dff  =  0,  and  c//  —  1  so  that  Case  4A  applies,  and  we  always 
get  a  stabilizing  solution.  From  (5.20)  and  (5.28)  we  find  the  closed  loop  system 

a;  =  -2x  +  a?2  +  ---  (5.29) 


which  is  indeed  (locally)  stable. 

R'om  these  examples  we  see  that  the  gap  between  the  stabilizability/detectability  conditions  and 
the  conditions  derived  herein  mainly  focuses  on  sufficiency  for  systems  that  have  dA  >  0  {A  has  a 
zero  constant  term),  and  necessity  in  all  other  cases  since  the  stabilizability/detectability  conditions 
do  not  address  necessity  at  all. 

5,5  Conclusion 

We  have  derived  necessary  and  sufficient  conditions  for  existence  of  analytic  stabilizing  solutions  to 
the  nonlinear  SDRE  regultor  problem  in  the  input-affine  single-state  case.  These  conditions  depend 
on  the  smallest  contributing  powers  of  x  in  the  polynomial  expressions  for  the  analytic  system  pa¬ 
rameters.  Although  the  single-state  nature  of  the  results  limits  their  applicability,  they  nevertheless 
oflfer  significant  insight  into  the  nature  of  SDRE  solutions  in  general  and  their  relationships  with 
LQR  theory.  In  the  succeeding  chapters  we  return  to  the  multistate,  multivariable  case  and  derive 
conditions  for  stability  which  extend  the  results  of  this  chapter. 
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VI.  Controllahility  Issues  in  SDRE  Control 


6. 1  Introduction 

In  this  chapter  we  return  to  the  general  multistate  case,  and  begin  to  fill  some  of  the  holes  currently 
existing  in  the  theory  of  the  SDRE-based  methods,  by  examining  controllability  issues  in  the  context 
of  the  state  feedback  regulator  problem.  These  issues  have  not  been  addressed  in  the  literature, 
and  have  significant  implications  for  global  stability  of  SDRE-based  nonlinear  control  algorithms. 
Recall  that  we  are  dealing  with  continuous  time,  state  feedback,  input-affine,  autonomous  nonlinear 
dynamic  systems  of  the  form 

X  —  a{x)  +  h{x)u^  a(0)  =  0 
h{x) 

,  M0)=0  (6.1) 

R{x)u 

with  state  vector  x  E  control  vector  u  €  7^"^,  penalized  variable  z  E  and  nonsingular  (for 
all  r)  control  penalty  matrix  function  R{x).  We  assume  a  and  h  are  globally  at  least  so  that  one 
can  obtain  a  factored  system  representation  of  the  form 

X  ~  A{x)x  -h  B{x)u 
H{x)x 
R{x)u 

Based  on  (6.2)  recall  that  our  suboptimal  solution  for  the  optimal  regulation  problem,  i.e.,  driving 
the  state  to  zero  while  simultaneously  keeping  the  cost  function 

fOO  POO 

J=  z  dt=  x^Q{x)x  +  u^R{x)u  dt  (6.3) 

Jo  Jo 

‘close’  to  its  optimal  value,  where  we  have  defined  R{x)  —  R^ {x)R{x)  >  0  and  Q{x)  =  H'^{x)H{x)  > 
0  V  a’,  is  given  by  setting 

u(x)  =  ~R~^{x)B^{x)P{x)x  (6.4) 
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where  P{x)  is  the  maximal,  stabilizing  solution  to  the  steady  state  continuous  time  state-dependent 
Riccati  equation  (SDRE) 

A^{x)P{x)  +  P{x)A{x)  -  P{x)B{x)R-^x)B'^{x)P{x)  +  Q(x)  =  0  (6.5) 

Now,  in  order  for  the  desired  solution  of  (6.5)  to  exist  for  all  x,  we  must  assume,  for  example, 
that  the  pairs  {A{x),  B{x)}  and  {H{x)^A{x)}  are  controllable  and  observable,  respectively,  for  all 
x^  where  we  employ  the  common  definitions  of  controllability  and  observability  from  linear  systems 
theory  [76].  Of  course,  less  restrictive  assumptions  such  as  stabilizability  and  detectability  for  all  x 
would  also  be  sufficient  to  ensure  global  existence  of  unique,  stabilizing  solutions  to  (6.5).  However, 
since  determining  stabilizability  requires  determining  controllability,  in  this  chapter  we  seek  first 
to  understand  the  relationship  between  the  'factored’  controllability  assumed  in  order  to  guarantee 
existence  of  solutions  of  (6.5),  and  the  true  nonlinear  controllability  of  the  system  (6.1).  Note  that 
although  SDRE  requires  observability  (detectability)  of  the  factorization  to  guarantee  existence  of 
Riccati  solutions,  true  nonlinear  observability  of  the  system  is  guaranteed  by  the  full  state  feedback 
assumption.  Thus,  comparison  between  nonlinear  and  factored  observability  will  not  be  included  in 
this  first  examination  of  the  state  feedback  case. 

6,2  Factored  versus  True  Controllability  of  Nonlinear  Systems 

Comparison  of  factored  versus  true  nonlinear  controllability  is  facilitated  by  considering  controllabil¬ 
ity  in  terms  of  the  dimension  of  invariant  locally  reachable  and  unreachable  spaces.  For  controllable 
systems,  the  dimension  of  the  locally  reachable  space  must  be  n,  the  dimension  of  the  state  space. 
Thus,  for  linear  time-invariant  (LTI)  systems 

X  =  Ax  -I-  Bu 

the  controllability  of  the  system  (6.6)  is  established  [41]  by  verifying  the  well-known  condition  that 
the  rank  of  the  controllability  matrix,  M^,  is  equal  to  n,  where 

AB  A^B  •••  A^-^B]  (6.7) 
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For  the  SDC  factored  system  (6.2),  the  above  test  for  controllability  generalizes  to  the  rank  test  on 
the  factored  controllability  matrix  function  Mcf(x) 

rank[Mcf(x)]  ~  rank[B{x)  A{x)B{x)  A^(x)B{x)  •••  A'^~^{x)B{x)]  =  x  (6.8) 

while  for  the  original  input-aiEne  nonlinear  system,  local  controllability  is  characterized  [33]  at  each 
X  in  terms  of  the  dimension  of  the  span  of  the  smallest  nonsingular  and  involutive  distribution, 
Ac(a^),  containing  the  columns  bi  of  B{x),  1  <  i  <  m,  and  invariant  under  a  and  the  6^.  This 
distribution  assigns  to  each  x  o,  vector  space,  an  open  subset  of  which  is  reachable  from  the 
given  point  by  using  piecewise  constant  inputs.  A  sufficient  condition  [33]  for  the  system  (6.1)  to  be 
locally  controllable  at  the  point  x  is  thus 

rank[^c{^)]  ~  n  (6.9) 

and  the  system  is  said  to  be  weakly  controllable  if  (6.9)  holds  for  all  x.  Note  that  if  we  desire  to 
indicate  that  (6.9)  holds  for  all  a;  in  a  set  5,  then  we  shall  say  the  system  is  weakly  controllable 
on  5.  Now,  we  recall  that  invariance  of  the  distribution  Ac  with  respect  to  a  means  that  the  Lie 
bracket  of  a  with  any  vector  field  r  G  Ac  is  a  vector  field  which  is  also  in  Ac-  This  fact  is  used  in 
[33]  to  suggest  a  recursive  algorithm  for  generating  Ac,  as  follows 

i.  Let  Ao  =  span{B)  ~  span{bi) 

ii.  Let  Ai  =  Ao  +  [a,  -f-  [&j,  fei],  1  <  ^  <  m,  1  <  i  <  m  where  [a,g]  indicates  the  Lie  bracket  of 

the  vector  fields  a  and  g  defined  by 

and  the  +  indicates  the  subspace  sum,  i.e.,  the  sum  of  the  spans. 

iii.  Let  A^  =  A/t-i  -f  [a,  dj]  +  [bi.dj],  1  <i  <m,  1  <  j  <  n  where  the  dj  form  a  basis  for  Aa:-i 

iv.  Terminate  when  A^+i  =  A^; 

It  is  shown  in  [33]  that  the  above  procedure  gives  Ac  =  Md  in  the  case  of  LTI  systems.  Also,  it 
is  proven  that  A^-i  generated  using  the  above  algorithm  always  equals  Ac  on  an  open  and  dense 
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subset  of  and  that  if  each  of  the  above  is  nonsingular,  then  An-i  =  A^  for  all  x.  Thus,  it 
takes  at  most  n-l  nontrivial  steps  of  the  above  recursive  procedure  to  give  a  picture  of  the  system 
controllability,  which,  although  not  total,  is  correct  on  most  of  the  set  of  interest. 

In  light  of  the  above  discussion  we  now  make  two  important  observations.  First,  if  a  system  fails 
to  pass  the  appropriate  controllability  rank  test,  then  the  uncontrollable  subspace  may  be  determined 
[33]  by  finding  the  annihilator  of  the  appropriate  matrix,  i.e.,  the  left  nullspace  of  Mcu  Mcf  or  Ac. 
Second,  the  theory  guarantees  that  the  controllable  and  uncontrollable  subspaces  as  determined 
from  Mci  and  Ac  are  invariant  subspaces  for  the  appropriate  systems,  while,  on  the  other  hand, 
the  controllable  and  uncontrollable  subspaces  determined  from  Mcf  hold  only  for  the  single  x  value 
being  considered.  Thus,  locally  uncontrollable  and  invariant  sets  might  or  might  not  exist  for  the 
factored  system,  and  further  analysis  would  be  warranted  to  determine  such. 

The  above  concepts  give  rise  to  controllable/uncontrollable  system  decompositions  [41],  [33].  For 
an  LTI  system  with  controllable  subspace  of  dimension  d,  the  system  (6.6)  may  be  partitioned 

—  ^11^1  +  A12X2  +  Biu 

^2  =  A22X2  (6.11) 

where  Xi  e  11^  is  in  the  controllable  subspace  Ci,  X2  E  is  in  the  uncontrollable  subspace  Ui, 

and  {^11,  Bi}  is  a,  controllable  pair.  For  the  nonlinear  system  (6.1)  with  controllable  subspace  of 
dimension  d,  we  similarly  may  write 


xi  =  ai{xi,X2)  +  6i(a;i,a;2)u 

(6.12) 

X2  =  a2{X2) 

(6.13) 

where  Xi  €  is  in  the  controllable  subspace  CnU  ^2  €  is  in  the  uncontrollable  subspace 

Uni,  and  (6.12)  satisfies  rank[/S.c]  —  d.  Finally,  (6.11)  -  (6.13)  allow  us  to  discuss  the  notion 
of  stabilizability.  Conceptually,  a  system  is  stabilizable  if  its  uncontrollable  part  is  stable.  For 
LTI  systems,  stabilizability  is  equivalent  to  A22  in  (6.11)  being  a  Hurwitz  matrix,  while  nonlinear 
stabilizability  requires  that  the  zero  dynamics  subsystem  (6.13)  be  stable.  For  the  SDC  factored 
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system  of  SDRE  control  (6.2),  we  have  a  decomposition  of  the  form  (6.11)  for  each  x  €  11^.  As 
discussed  above,  this  gives  a  pointwise  description  of  controllable  and  uncontrollable  states.  However, 
the  fact  that  the  decomposition  changes  in  general  from  point  to  point  without  guaranteed  invariance 
greatly  complicates  stability  analysis  based  on  the  decomposed  subsystems. 

6,3  Theorems  and  Examples 

With  some  minor  extensions,  the  discussion  of  Section  6.2  allows  us  to  prove  some  elementary 
theorems,  the  first  of  which  is  the  lack  of  a  general  equivalency  between  factored  and  true  nonlinear 
controllability. 

Theorem  6.3.1  Consider  the  system  (6.1)  with  a{x)  and  h(x)  assumed  to  be  functions^  so  that 
(6.1)  may  he  written  as  in  (6.2).  Assume  the  pair  {A(a;),P(a:)}  is  controllable  for  all  so  that 
(6.8)  holds.  Then  the  system  (6.1)  is  not  necessarily  weakly  controllable. 

Proof:  The  proof  is  by  counterexample.  Consider  the  system 

Xx  =  X1X2  +  X2 

±2  =u  (b.l4) 

Choose 

X2  1 

A{x)  =  (6.15) 

0  0 

and  observe  that  b{x)  =  [0  1]^.  Then 

0  1 

Mcf  =  [b  A{x)b]  =  (6.16) 

1  0 

so  that  Mcf  is  full  rank  for  all  x,  and  the  assumptions  of  the  theorem  hold.  Let  us  now  construct 
Ac  according  to  the  recursive  procedure  given  in  Section  6,2.  We  have 

0 

Ao  =  span[b]  =  span  (6.17) 

1 
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Ai  —  Ao  -h  [ft,  b]  -j-  [bj  b] 


(6.18) 


Now,  since  [6,  &]  —  0  for  any  vector  field,  and  &  is  a  constant  vector  so  that  ||  ~  0,  we  have 


0(1 

Ai  =  Ao  +  [ft,  b]  —  b-\-  —b  = 


0 

X2  a;i  + 1 

0 

4- 

1 

O 

o 

1 

(6.19) 


Ai  =  span 


(6.20) 


so  that 

r 

0  xi  +  l 
1  0 

which  equals  Ac  on  at  least  an  open  and  dense  subset  of  'R?.  Performing  one  more  iteration  of  the 
procedure  we  find  (noting  that  di  =  b) 


A2  —  Ai  +  [a,  b]  -f  [a,  ^2]  -H  [h,  b]  4-  [h,  6^2] 


(6.21) 


where 


^2  — 


Xi  -\-l 

0 


(6.22) 


Now,  since  [b,  6]  —  0  and  [a,  b]  is  already  in  Ai,  we  need  only  compute  the  other  two  Lie  brackets  in 
(6.21).  It  is  easily  established  that 


[ft,  d2 


1  0 
0  0 


X1X2  +  X2 

0 


X^X2  X2  —  X\X2  -  X2 
0-0 


and 


[b,d2]  = 


1  0 
0  0 


X2  a;i  + 1 

0  0 

0 
0 


aij  + 1 

0 

(6.23) 


so  that  we  have  A2  =  Aj  =  Ac  and  the  procedure  terminates.  Thus,  dearly,  at  x\  = 
ranA;[Ac]  =  1  ^  n  =  2,  so  that  (6.9)  fails  to  hold  and  the  theorem  is  proved. 


(6.24) 
-1,  we  have 


Pi'om  the  proof  of  the  previous  theorem  we  see  that  it  is  possible  for  a  state  dependent  factoriza¬ 
tion  to  hide  the  existence  of  an  uncontrollable,  invariant  set  (the  set  of  all  x  such  that  —  1 
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for  (6.14)).  This  fact  is  not  mentioned  in  [13],  nor,  it  appears,  has  it  historically  been  well  known 
when  SDRE  type  methods  were  previously  suggested.  In  fact,  in  [70],  a  state-dependent  regulation 
algorithm  is  given,  and  the  factorization  (6.15)  is  recommended  as  a  better  factorization  for  (6.14) 
than  the  choice 


A{x)  = 


0 


(6.25) 


0  0 


because  (6.15)  allows  solution  for  the  control  at  all  x,  while  (6.25)  does  not.  More  recently,  [21] 
suggests  an  SDRE-like  regulation  algorithm  accompanied  by  the  assertion  that  it  works  well  quite 
often,  but  with  virtually  no  theoretical  basis  to  support  the  claim,  or  to  indicate  when  the  method 
might  fail. 

Upon  comparing  Mcf  and  Ac  for  (6.14),  it  can  be  seen  that  the  two  differ  only  in  their  second 
columns,  because  the  second  column  of  Me/  is  given  by  A{x)b,  while  the  second  column  of  Ac 
is  J{x)b,  where  J  is  the  Jacobian  of  a,  i.e.,  J{x)  =  This  observation  allows  us  to  prove  an 
equivalency  relationship  between  factored  and  true  nonlinear  controllability  for  the  special  case  of 
2nd-order  systems  with  constant  B  matrices,  as  formalized  below. 


Theorem  6.3.2  Consider  the  system  (6.1)  written  as  (6.2)  with  n  —  2,  and  let  B  be  a  constant 
matrix.  Also,  assume  that  A{x)  is  chosen  such  that  J{x)B  =  kA{x)B  V  x,  where  0^  k  e  TZ.  Then, 
if  the  factorization  (6.2)  is  controllable  for  all  x,  the  system  (6.1)  is  weakly  controllable  on  an  open 
and  dense  subset  of'R?.  Conversely,  if  (6.1)  is  weakly  controllable  on  TZ^ ,  then  (6.2)  is  controllable 
for  allx. 


Proof:  The  proofs  follow  by  simply  computing  and  checking  the  rank  of  Mcf  and  Ac  under  the 

given  assumptions.  For  the  first  claim,  with  (6.2)  being  controllable  for  all  x,  (6.8)  implies 

rank[Mcf(x)]  —  rank[B  A{x)B]  “  2  V  a;  (6.26) 

Now,  for  (6.1),  on  an  open  and  dense  subset  V  of  we  have 

Ac(a;)  =  Ai{x)  ~  B  +  [a,  hi]  -jr  [bj,  bi],  1  <  i  <  m,l  <  j  <m  (6.27) 
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Now,  since  the  hi  are  constant  vectors,  we  have  —  0  V  and  [a, =  ~Jbi,  so  that 

Ac(a;)  =  [B  J{x)B]  -  [B  kA{x)B]  (6.28) 

which  is  guaranteed  to  be  full  rank  for  all  x  since  Mcf{x)  is  full  rank  and  k  ^0,  The  second  claim 
follows  by  the  reverse  logic  path:  starting  with  Ac{x)  =  [B  J{x)B]  assumed  full  rank  for  all  x,  and 
substituting  for  J{x)B  to  obtain  (6.28),  this  implies  Mcf{x)  must  be  full  rank  for  all  x,  since  the 
nonzero  scaling  k  may  be  pulled  out  of  the  second  column  of  Me/.  ■ 

We  note  that  the  forward  claim  could  be  strengthened  to  weak  controllability  on  all  of  TZ^  by 
requiring  the  additional  assumption  that  Ac{x)  =  Ai{x)  V  x,  and  not  just  on  an  open  and  dense 
subset  of  'R},  In  Theorem  6.3.2  we  restrict  attention  to  2nd-order  systems,  because  for  higher- 
order  systems  we  will,  in  general,  need  to  perform  more  iterations  on  (as  many  as  —  1  total 
before  we  know  Ac  on  an  open  and  dense  subset),  which  will  require  repeated  Lie  brackets.  As 
shown  below,  the  first  Lie  bracket  computation  yields  the  second  entry  of  Ac,  which  differs  from 
the  second  entry  of  Mcf  by  only  a  single  additive  term,  due  to  the  fact  that  B  is  constant.  The 
divergence  between  succeeding  entries  in  the  two  matrix  functions  will  increase  as  more  Lie  brackets 
are  required,  and  would  be  significantly  greater  even  at  the  first  Lie  bracket  computation  if  B  were 
not  constant.  We  now  illustrate  these  concepts  by  computing  the  needed  Lie  brackets,  observing 
that  since  a{x)  =  A[x)x^  then 

BA 

J{x)  -  A{x)  A^x  =  A{x)  4-  Ta{x)x  (6.29) 

ax 

where  the  partial  derivative  term  Ta  is  a  third  rank  tensor,  and  thus 

[a,  hi]  =  Jhi  =  Abi  +  TAxbi  (6.30) 

Note  first  that  if  bi  were  not  constant,  then  the  Lie  bracket  [a,  bi]  would  include  another  term  from 
the  partial  of  bi  with  respect  to  x.  Also  note  that  from  (6.30),  for  the  conditions  of  the  theorem 
to  hold,  we  need  Ta{x)xB  =  (  J  -  A)B  —  {k  -  1)AB  where  k  cannot  equal  0,  and  thus  we  cannot 
have  Ta{x)xB  =  —AB  for  any  x  if  we  require  the  second  entry  in  the  controllability  matrices  to 
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provide  some  of  the  needed  rank.  We  also  see  that  the  theorem  requires  that  AB  =  0  if  and  only 
if  JB  —  0,  and  this  is  also  a  necessary  condition  if  B  is  not  full  rank.  Now,  if,  for  example,  n  —  3, 
then  a  second  Lie  bracket  computation  needed  to  compute  A2  gives 

[a,Jbi]^^a-jHi  (6.31) 

and  we  see  that  although  the  assumptions  of  the  theorem  give  A^B  in  the  range  of  J^B,  there  is 
still  potential  mismatch  between  Mcf  and  Ac  due  to  the  contribution  from  the  term  involving  the 
partial  of  Jbi  with  respect  to  x.  Finally,  we  note  that  the  conditions  of  the  theorem  are  sufficient, 
but  far  from  necessary.  It  is  possible  for  both  Mcf  and  Ac  to  be  full  rank  without  AB  being  in  the 
range  of  JB  alone  as  the  theorem  requires,  as  long  as  both  AB  and  JB  provide  the  remainder  of  a 
spanning  set  of  11^  that  is  not  provided  by  B. 

The  above  expression  (6.29)  relating  the  Jacobian  J  of  a  in  the  original  nonlinear  system  (6.1)  to 
the  factorization  A  in  (6.2)  allows  us  to  draw  an  additional  conclusion  regarding  factored  and  local 
nonlinear  controllability,  as  stated  in  the  following  theorem. 

Theorem  6.3.3  Consider  the  system  (6,1)  written  as  (6,2) ^  and  assume  (6,8)  holds.  Then  the 
system  (6.1)  is  weakly  controllable  on  some  local  neighborhood  of  the  origin. 

Proof:  If  (6.8)  holds  for  all  x,  then  clearly  (6.8)  holds  for  a:  =  0.  Now,  observe  that  (6.29)  implies 
that  J(0)  =  -A(O),  so  that  (6.8)  gives 

rank[Mcfi0)]=-rank[B{0)  JiO)B{x)  J^{0)B{0)  •••  J^-\0)B{0)]  =  n  (6.32) 

Now,  it  is  well  known  [39,  62,  68]  that  in  some  local  neighborhood  of  the  origin,  the  behavior  of 
the  nonlinear  system  (6.1)  is  governed  by  that  of  its  linearization  about  the  origin  (provided  the 
linearization  is  nontrivial),  which  is  an  LTI  system  given  by 

X  ^  J{0)x  -h  B{Q)u  (6.33) 

Clearly,  (6.32)  implies  that  (6.33)  is  controllable,  so  that  the  nonlinear  system  (6.1)  is  weakly  con- 
trollable  on  the  set  where  the  linearization  (6.33)  dominates,  and  the  theorem  is  proven.  ■ 
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The  previous  three  theorems  dealt  with  an  equivalence  relationship  between  Mcf  and  when 
the  computation  of  Ac  required  computation  beyond  Aq.  If  the  full  rank  requirement  is  met  by 
the  B  matrix  itself,  then  we  clearly  have  equivalence  between  factored  and  nonlinear  controllability 
without  any  assumptions  on  A{x)  or  on  the  dimension  of  the  state,  as  formalized  in  the  following 
theorem. 

Theorem  6.3«4  Consider  the  system  (6.1)  written  as  (6.2)  ,  and  assume  B{x)  has  rankn  for  all 
X.  Then  (6.2)  is  controllable  for  all  Xj  and  the  system  (6.1)  is  weakly  controllable  on  TIP' . 

Proof:  The  proof  follows  trivially  from  the  construction  of  Me/  and  Ac,  noting  that  B[x)  appears 
as  the  first  component  of  both  matrix  functions,  and  itself  satisfies  the  rank  requirement.  ■ 

It  is  important  to  note  that,  for  the  assumptions  of  Theorem  6.3.4  to  hold,  we  must  have  m  >n 
(at  least  as  many  controls  as  states).  We  also  observe  that  Theorem  6.3.4  includes  the  special  case 
of  B  equal  to  a  constant,  full  rank  matrix.  This  fact  is  exploited  in  Chapter  7.  We  now  illustrate 
the  above  theorems  by  means  of  some  examples. 

Example  1 

Consider  the  system 


^1  =  ^2 
X2  =  XiX2  -f  u 


We  have 


X2 


X1X2 


,  6  = 


(6.34) 


(6.35) 


and  b  is  not  full  rank  for  any  x,  so  that  Theorem  6.3.4  may  not  be  used.  However,  we  have 

0  1 


J  = 


X2  Xi 


(6.36) 


and 


A,  =  Ai  = 


0  1 
1  X\ 


(6.37) 
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which  has  rank  2  for  all  x.  Now,  choose 


Then 


A{x)  = 


0  1 
0  xi 


(6.38) 


A{x)B  = 


1 


Xi 


JB'ix 


(6.39) 


so  that  the  conditions  of  Theorem  6.3.2  are  satisfied.  Checking  the  factored  controllability  matrix 
we  find 


Me  fix) 


0  1 
1  Xi 


=  Acix)  V  X 


(6.40) 


and  we  see  that  the  original  system  is  weakly  controllable  on  TZ^,  while  the  factored  system  is 
controllable  for  all  x  £  TZ^  as  well.  On  the  other  hand,  choose 


Aix)  = 


X2  1-  Xi 


0  xi 


which  is  easily  verified  to  yield  Aix)x  =  a(a:).  However, 


(6.41) 


Mefix)  — 


0  1  —  xi 

1  Xi 


(6.42) 


which  clearly  loses  rank  at  =  1.  Given  the  analysis  above,  the  first  choice  of  for  this  example 
is  preferable,  because  it  guarantees  global  existence  of  the  control.  Prom  this  example  we  thus  see 
that  even  when  the  original  system  is  weakly  controllable,  care  must  be  taken  when  choosing  the 
factorization,  as  both  poor  and  good  choices  from  an  implementation  standpoint  may  exist. 

In  the  next  example  we  illustrate  the  nonnecessity  of  the  AB  =  kJB  condition  as  discussed 
above. 

Example  2 

Consider  the  system 


±1  —  X2 

±2  ~  x\-\-x\-\-u  (6.43) 
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We  have 


X2 

0 

a  “ 

xf  +  X2 

.  b  = 

1 

so  that  again  Theorem  6.3.4  may  not  be  invoked.  Computing  the  Jacobian  of  a  we  find 


(6.44) 


J  = 


0 


1 


2xi  2x2 


so  that 


Ac  =  Ai 


which  has  rank  2  for  all  x.  Now,  choose 


0  1 
1  2X2 


(6.45) 


(6.46) 


A{x)  = 


0  1 

Xi  X2 


(6.47) 


Then 


A{x)B  = 


X2 


(6.48) 


which  does  not  equal  a  multiple  of  JB  for  any  x.  However,  checking  the  factored  controllability 
matrix  we  find 


Mcf{x)  = 


0  1 
1  X2 


(6.49) 


which  is  full  rank  for  all  a;,  and  again  we  see  that  the  original  system  is  weakly  controllable  on 
V?,  while  the  factored  system  is  controllable  for  all  x  E  Thus,  even  though  the  conditions  of 
Theorem  6.3,2  are  not  satisfied,  [1  2x2]^  and  [1  ^2]^  both  provide  vector  functions  which,  together 
with  &  =  [0  1]^,  span  TZ^  for  any  value  of  x. 

Finally,  we  return  to  the  counterexample  in  the  proof  of  Theorem  6.3.1,  and  see  how  the  discussion 
of  necessary  conditions  for  equivalence  of  factored  and  true  nonlinear  controllability  in  2nd-order 
systems  with  constant  B  matrices  following  Theorem  6.3.2  comes  into  play. 
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Example  3 

Consider  the  system 


Xi  =  X1X2  +  X2 
X2~U 


(6.50) 


We  have 


a  — 


b  = 


(6.51) 


X1X2  +  X2 

0 

so  that  rank{B)  -  1  V  a;  and  again  Theorem  6.3.4  may  not  be  invoked.  Computing  the  Jacobian  of 
a  we  find 


J  = 


xi  ail  + 1 

0  0 


(6.52) 


so  that 


A,= 


0  Xi  1 
1  0 

which  has  rank  1  at  a;i  =  —  1  as  we  saw  before.  Now,  also  as  before  choose 


(6.53) 


A{x)  = 


X2  1 
0  0 


(6.54) 


Then 


A(x)B  = 

which  gave  the  factored  controllability  matrix 

Afcf{x)  = 


1 

0 

0  1 
1  0 


(6.55) 


(6.56) 


which  is  full  rank  for  all  x.  However,  we  see  that  kJB  does  not  equal  AB  iox  x=  -1  for  any  kj^O. 
Also,  TaxB  =  (J  —  A)B  =  —AB  when  x^  =  —1,  and  we  have  AB  0  when  JB  =  0,  both  of 
which  we  noted  were  necessary  conditions  for  equivalency  of  the  controllability  tests  if  B  was  not 
full  rank.  Thus,  we  observe  the  discrepancy  between  the  factored  system  being  controllable  for  all 
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X  G  while  the  original  system  fails  to  be  nonlinearly  controllable  on  X\  —  -1.  This  highlights 
the  reverse  of  what  was  found  at  the  end  of  Example  1.  There,  we  saw  existence  of  nonglobally 
controllable  factorizations  for  a  weakly  controllability  system,  while  here  we  observe  existence  of 
globally  controllable  factorizations  when  the  original  system  is  not  weakly  controllable.  Finally,  this 
example  illustrates  the  potential  weakness  of  Theorem  6.3.3.  Clearly,  the  controllability  equivalence 
based  on  linearization  does  not  hold  in  this  example  beyond  a  ball  of  radius  one  centered  at  the 
origin.  By  replacing  the  first  element  of  a  in  (6.51)  with  xiX2  +  kx^^  with  k  eTZ,  and  decreasing  the 
absolute  value  of  A:,  we  can  construct  an  example  for  which  the  conclusion  of  Theorem  6.3.3  holds  on 
an  arbitrarily  small  neighborhood  of  the  origin.  Thus,  the  study  of  global  as  opposed  to  just  local 
controllability  equivalence  is  indeed  well-motivated. 

6.4  Summary  and  Conclusions 

We  have  shown  that,  in  general,  nonlinear  controllability  of  input-affine  systems  (6.1)  and  control¬ 
lability  of  a  state-dependent  factorization  (6.2)  of  such  systems  in  terms  of  linear  type  tests  are  not 
equivalent  concepts.  We  have  shown  that  controllability  of  a  state-dependent  factorization  for  all 
X  is  sufficient  to  guarantee  weak  controllability  on  the  neighborhood  of  the  origin  dominated  by 
the  linearized  dynamics,  and  we  have  shown  that  if  ^(a;)  is  full  rank  for  all  x^  then  the  conditions 
for  both  controllability  tests  are  simultaneously  met,  regardless  of  the  choice  of  A  matrix  factor¬ 
ization  or  dimension  of  the  state  vector.  We  have  also  shown  sufficient  conditions  for  equivalency 
in  the  special  case  of  second-order  systems  with  constant  B  matrices,  which  may  be  extended  to 
necessary  conditions  when  B  is  not  full  rank.  For  higher  order  systems,  or  for  systems  with  noncon¬ 
stant,  non-full  rank  B  matrix  functions,  we  have  shown  that  conditions  guaranteeing  equivalency 
become  increasingly  complex  to  meet,  due  to  differences  between  the  Lie  brackets  which  characterize 
nonlinear  controllability  and  the  A^B  products  in  the  factored  controllability  test. 

While  factored  controllability  for  all  x  is  sufficient  to  guarantee  global  well-posedness  of  SDRE- 
based  control  algorithms,  it  is  in  general  not  a  sufficient  assumption  to  guarantee  true  controllability 
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of  the  nonlinear  system  outside  some  possibly  small  neighborhood  of  the  origin,  a  fact  which  has 
significant  ramifications  for  global  stability  of  SDRE  control.  In  fact,  (6.12)  and  (6.13)  indicate  that, 
regardless  of  how  well  the  SDRE  algorithm  operates,  it  can  only  ailect  the  portion  of  the  state  which 
is  nonlinearly  controllable.  Thus  a  necessary  condition  for  stability  of  the  SDRE  regulator  is  that 
the  nonlinear  system  be  nonlinearly  stabilizable,  as  discussed  and  defined  in  Sections  4.7  and  6.2. 
This  may  be  sufllcient  if  only  regulation  is  to  be  attempted,  but  full  nonlinear  controllability  may 
be  required  for  trajectory  tracking  or  model  following  type  tasks. 

Finally,  in  this  chapter  we  have  been  concerned  only  with  the  controllability  part  of  the  control- 
lability/observability  conditions  assumed  for  well-posedness  of  the  SDRE  algorithm,  and  our  study 
of  factored  and  true  nonlinear  controllability  was  motivated  by  the  desire  to  analyze  and  predict 
possible  success  of  the  state  feedback  SDRE  regulation  algorithm  in  guaranteeing  global  aysmptotic 
stability  of  the  closed  loop.  Thus,  it  is  fitting  that  we  conclude  by  making  some  brief  comments  on 
the  observability  part  of  the  well-posedness  assumptions.  If  we  limit  ourselves  to  the  state  feedback 
case,  we  have  assumed  full  nonlinear  observability,  so  there  would  be  no  worry  about  factored  ob¬ 
servability  hiding  truly  unobservable  modes.  Factored  observability  will,  however,  still  play  a  part  in 
stability  analysis  of  the  regulator,  since  pointwise  unobservable  modes  may  not  receive  any  control 
action  [41].  These  issues  are  fully  addressed  in  Chapter  9.  In  the  output  feedback  case,  additional 
theory  of  factored  versus  nonlinear  observability,  analagous  to  what  has  been  derived  herein,  will 
be  required  for  a  full  understanding  of  the  SDRE-based  methods.  As  mentioned  previously,  these 
issues  are  not  addressed  in  this  dissertation,  but  are  proposed  as  avenues  of  further  research. 
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VIL  Stability  of  Systems  with  Full  Rank  Constant  B  Matrices 


7. 1  Introduction 

In  this  chapter  we  build  on  the  results  of  Chapter  6,  in  particular  making  use  of  the  equivalence  of 
factored  and  nonlinear  controllability  for  systems  with  full  rank,  constant  B  matrices,  as  given  in 
Theorem  6.3.4.  We  still  consider  continuous  time,  state  feedback,  input-affine,  autonomous  nonlinear 
dynamic  systems  described  by  6.1  and  6.2,  but  we  now  assume  we  have  state  and  control  vectors 
x,u  e  penalized  variable  2:  €  7?.^^,  and  nonsingular  constant  control  penalty  matrix  R,  We 
assume  that  a  and  h  are  real- valued  functions  of  x  on  so  that  A  and  H  are  chosen  to  be  (at 
least)  continuous  functions.  We  further  assume  that  b{x)  =  B  is  a  full  rank  constant  matrix  and  that 
H'^{x)H{x)  is  positive  definite  for  all  x  {H{x)  is  globally  nonsingular).  Under  these  conditions  we 
clearly  have  that  the  pairs  {A{x),B]  and  {H{x),A{x)]  are  controllable  and  observable,  respectively, 
for  all  where  we  employ  the  common  definitions  of  controllability  and  observability  from  linear 
systems  theory.  Finally,  we  assume  that  x  =  0  is  the  only  open  loop  equilibrium  of  (6.1).  Using  these 
assumptions  we  will  now  show  that  the  continuous  time  state  feedback  nonlinear  SDRE  regulator 
yields  a  closed  loop  system  which  is  globally  asymptotically  stable.  We  also  show  that  the  assumption 
of  a  single  open  loop  equilibrium  at  the  origin  may  be  relaxed  by  placing  some  constraints  on  the 
magnitude  and/or  structure  of  the  state  weighting  matrix  function  Q{x)  =  H'^{x)H[x). 

7.2  Control  Algorithm 

Recall  that  the  control  is  given  by 

u{x)  =  -R-^B'^P{x)x  (7.1) 

where  P{x)  is  the  positive  definite  maximal  solution  to  the  steady  state  continuous  time  state- 
dependent  Riccati  equation  (SDRE) 

A^{x)P{x)  +  P{x)Aix)  -  P{x)BR-^B'^P{x)  +  Q{x)  =  0  (7.2) 
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which  is  guaranteed  to  exist  for  all  x  due  to  the  global  controllability /observability  of  the  SDC 
factorizations.  Note  that  in  (7,1)  and  (7.2)  i?  is  a  constant  matrix  R  =  R^R  >  0,  and  B  is  also  a 
constant  matrix,  but  P  remains  a  function  of  x  due  to  the  a;-dependency  of  A  and  Q, 

7,3  Global  Asymptotic  Stability  via  the  Direct  Method  of  Lyapunov 

We  now  prove  global  asymptotic  stability  of  the  closed  loop  system  resulting  from  the  SDRE  continu¬ 
ous  time  control  algorithm  outlined  in  Section  7,2.  The  method  of  proof  involves  standard  Lyapunov 
analysis  [36]  in  a  transformed  set  of  coordinates,  and  is  formalized  below. 

Theorem  7*3.1  Consider  the  system  (6.1)  with  a{x)  and  h{x)  assumed  to  be  functions  and 
such  that  a(0)  —  0,  h{0)  =  0.  Assume  further  that  a{x)  =  0  ^  x  —  Oj  rank{B)  =  n  and 
>  0  V  X  where  h{x)  =  H{x)x,  Also,  assume  R  in  (6.1)  is  constant  and  nonsingular. 
Then  application  of  the  SDRE  nonlinear  regulator  control  algorithm  defined  by  (7.1)  and  (7.2)  to 
(6.1)  yields  a  closed  loop  system  which  is  globally  asymptotically  stable. 

Proof:  As  explained  in  Section  7.1,  the  assumptions  on  a{x)  and  h{x)  guarantee  that  (6.1)  may 

be  written  in  the  form  of  (6.2),  where  A{x)  is  nonsingular  for  all  x  since  a{x)  —  A(x)x  —  0  only  for 
X  0.  Consider  now  the  change  of  coordinates  defined  by 

q=B-^x  (7.3) 

where 

B  =  BR-^  (7.4) 

and  note  that  (7.3)  and  (7.4)  define  a  globally  valid  static  linear  diffeomorphism  since  5  is  a  constant 
matrix,  B  has  full  rank,  and  R  is  nonsingular.  Consider  now  the  Lyapunov  function 

Viq)  =  \q'^q  (7.5) 

which  is  positive  definite  and  decrescent,  and  also  radially  unbounded.  Taking  the  derivative  of  (7.5) 
we  find 

V=\fq  +  \q^q  (7.6) 
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•where  from  (7.3)  we  have 


q  =  B~^x 

Now,  substitutiag  (7.1)  into  (6.2)  and  using  we  have 

X  —  A{x)x  —  BR~^B^P(x)x  =  A{x)x  -  BR~^R~^B^P{x)x 
so  that  using  (7.4)  we  obtain 

X  =  A{x)x  -  BB^P(x)x 

Premultiplying  (7.9)  by  B~^  and  substituting  into  (7.7)  we  find 

q  =  B~^A{x)x  —  B^  P{x)x 
Now,  if  we  use  the  fact  that  x  =  Bq  and  define 

A{q)  =  B-^A{x)B 


(7.7) 


(7.8) 


(7.9) 


(7.10) 


(7.11) 


and 

P{q)  =  B'^P{x)B 

then  (7.10)  may  be  written 


q  =  A(q)q  -  P(q)q  =  [A{q)  -  P(q)]q 


(7.12) 


(7.13) 


Now,  observe  that  since  (7.11)  just  defines  a  similarity  transformation,  A{q)  has  the  same  eigenvalues 
as  A{x)^  and  thus  A{q)  is  nonsingular  for  all  q.  Likewise  [42],  the  congruence  transformation  defined 
by  (7.12)  preserves  positive  definiteness  and  symmetry  of  P{x)  in  P(^),  where  the  positive  definite¬ 
ness  of  P{x)  is  guaranteed  by  controllability  and  observability  of  (6.2)  for  all  x  [76].  Substituting 
(7.13)  into  (7.6)  we  thus  find 


V  -q^  [^(^)  -h  A^{q)  -  2P{q)\  q q^ 


A{q)+Ar{q) 


q  -  q^P{q)q 


(7.14) 


Now,  using  Rayliegh’s  quotient  [57]  we  have 

\A{q)  +  A^{q) 


q  <  X 


Aiq)  +  A'^(q) 


T 

Q  q 


(7.15) 
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q’'P{q)q  >  A[P(q')]  q^q 


(7.16) 


where  A  and  A  denote  the  maximum  and  minimum  eigenvalues  of  a  matrix,  respectively.  From  (7.16) 
we  have  -q^P{q)q  <  -K[P{q)]q^Q  so  that  we  may  write 


T 

Q  Q 


(7.17) 


Finally,  recognizing  that  A  j  is  the  matrix  measure  of  A  with  respect  to  the  vector  (Euclidean) 

2-norm  [69],  we  define  it  as 

\A{q)+A'^{qy 


fJ‘2[A{q)]  =  A 


(7.18) 


so  that  (7.17)  becomes 


V  <  {/J.2[A{q)]  -  X[P{g)])g'^q 


(7.19) 


We  now  seek  to  prove  that  the  right  hand  side  of  (7.19)  is  always  negative,  and  to  do  so  we  turn  to 
properties  of  the  algebraic  Riccati  equation.  Observe  that  with  A{x)  =  BA(q)B~^  and  — 

BB^,  the  continuous  time  SDRE  (7.2)  becomes 


[JBA{q)B-YP(^)  +  P{x)[BA{q)B-^]  -  P(x)BB'^P{x)  A  Qix)  =  0  (7.20) 


Thus,  premultiplying  by  B^  and  postmultiplying  by  B,  (7.20)  becomes 


A'^{q)B'^Pix)B  +  B'^P{x)BA{q)  -  B'^P{x)BB'^ P{x)B  +  B'^Q{x)B  =  0  (7.21) 


SO  that  by  defining 


Q{q)  =  B'^Q{x)B 


and  recalling  (7.11)  and  (7.12)  we  may  write  (7.21)  as 


A'^iq)Piq)  +  P{q)A{q)  -  P{q)P(q)  +Q(q)  =  0 


(7.22) 


(7.23) 


Note  that  with  H'^{x)H(x)  =  Q{x)  >  0  and  B  full  rank  we  have  Q{q)  >  0  V  <?.  Now,  we  make  use  of 
a  lower  bound  on  the  minimum  eigenvalue  of  the  solution  of  the  continuous  time  algebraic  Riccati 
equation 

A^P  +  PA-  PBB^P  +  (5=0  (7.24) 
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given  in  [49],  which  is 


m> 


-fJ^A  +  +  KBB'r)^\\{{A^A)-^Q} 


(7.25) 


where 


(7.26) 


and  /i2(^)  is  as  defined  in  (7.18).  Note  that  for  (7.25)  to  be  valid,  we  must  have  > 

0  and  (A^A)"^  must  exist.  Observe  that  from  (7.23)  we  have  B{q)B^ [q)  =  7  so  that  clearly 
A[jB(g)jB^(g)]  =  X[B{q)B^ (q)]  =  1,  and  the  first  condition  above  is  satisfied.  As  discussed  above  we 
also  have  A{q)  nonsingular  for  all  g,  so  that  the  second  condition  is  also  satisfied,  and  thus  (7.25) 
may  be  applied  to  (7,23).  Finally,  we  clearly  have  /J>Aiq)  >  —fi2{A(q))  so  that  (7.25)  gives 

A[^’(9)]  >  7(2 [^(?)]  +  +  l^l[A{q)]M.W{(l)A{q)]-^Q{q)}  (7.27) 


Now,  if  //2[A(g)]  ^  0,  then  clearly  (7.27)  implies 


X[P{q)]  >  /i2[A(g)]  (7.28) 

so  that 

fi2[A{q)]-X[P{q)]<0  (7.29) 

and  from  (7.19)  we  see  that  F  <  0.  If  /i2[A{q)]  —  0,  then  (7.19)  becomes 

<  -MP{q)]q^q  (7.30) 

and  once  again  we  see  that  17  <  0.  Finally,  the  positive  definiteness  of  P(q)  for  all  q  guarantees  that 

q^ P{q)q  vanishes  only  at  the  origin,  and  this  completes  the  proof.  ■ 

We  note  that  in  the  above  proof,  it  was  not  necessary  that  the  control  be  applied  to  the  system 
in  transformed  q  coordinates.  The  control  was  applied  in  the  original  set  of  x  coordinates,  and 
only  the  stability  analysis  was  performed  in  the  q  coordinates.  One  could,  of  course,  first  transform 
the  system  to  have  identity  B  matrix  (including  the  effects  of  7^),  and  the  above  result  would  still 
hold.  Indeed,  this  observation  suggests  that  we  could  obtain  a  globally  asymptotically  stabilizing 
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SDRE  regulator  for  systems  with  nonconstant  but  full  rank  B  matrix  functions  for  all  x,  by  defining 
u{x)  =  B~^{x)v[x),  and  using  the  SDRE  regulation  algorithm  to  find  the  new  control  v. 

In  seeking  to  apply  Theorem  7.3.1,  we  may  find  the  nonsingularity  constraint  on  A  rather  re¬ 
strictive,  and  we  thus  seek  a  means  of  relaxing  this  requirement.  This  may  be  accomplished  by 
defining 

N,~{qeTr\  det[A{q)]  =  0}  (7.31) 

i.e..  Ns  is  the  set  of  all  points  in  g-space  where  A{q)  is  singular,  and  making  various  assumptions 
about  the  nature  of  iV^.  We  now  present  several  theorems  in  this  vein. 

Theorem  7.3,2  Assume  all  the  conditions  given  in  Theorem  7.3.1  hold  except  for  a{x)  =  Q  ^  x  = 
0.  Define  Ng  as  in  (7.31)  and  assume  that  for  all  q  G  Ng^  that  A{q)q  =  B~^A{x)x  =  0.  Then  the 
closed  loop  system  defined  by  (6.2),  (7.1),  and  (7.2)  is  globally  aymptotically  stable. 

Proof:  With  V  defined  as  in  (7.5),  the  proof  that  V^(^)  <  0  for  all  q  not  in  Ng  follows  as  per  the 
proof  of  Theorem  7.3.1.  For  q  E  since  A{q)q  ^  q'^A'^{q)  vanishes,  we  have  V  =  -2q’^P{q)q  <  0 
by  the  global  positive  definiteness  of  P(^).  Thus,  we  have  a  globally  positive  definite,  decrescent, 
and  radially  unbounded  Lyapunov  function  with  globally  negative  derivative,  so  that  the  theorem 
is  proven.  ■ 

The  point  of  the  above  theorem  is  that  it  is  not  the  open  loop  equilibrium  points  that  cause  a 
potential  problem  for  stability,  since  F  <  0  at  such  points.  It  is  instead  the  points  at  which  A  is 
singular  but  the  state  derivative  does  not  vanish  that  are  potentially  troublesome  for  the  proof.  If 
we  therefore  define  the  set 


N  =  {qE  n^\  det[A{q)]  =  0  and  A{q)q  ^  0}  (7.32) 

it  is  on  this  set  which  we  must  make  additional  assumptions  to  guarantee  stability,  as  we  do  in  the 
following  theorems. 


7-6 


Theorem  7*3,3  Assume  all  the  conditions  given  in  Theorem  7.3,1  hold  except  for  a{x)  =  0  a:  = 
0.  Define  N  as  in  (7.32)  and  assume  N  is  bounded.  Then  all  trajectories  of  the  closed  loop  system 
defined  by  (6.2),  (T^Vi  globally  bounded. 

Proof:  The  conclusion  follows  trivially  from  the  existence  of  a  globally  positive  definite,  decrescent, 
and  radially  unbounded  Lyapunov  function  (7.5)  which  has  negative  derivative  outside  of  a  compact 
set  containing  the  origin  [40] .  ■ 

If  we  now  define  the  set 


E  =  {qe  R^\  q'^[A{q)  +  (q)  -  2P(q)]q  =  0}  (7.33) 


and  make  an  assumption  on  E  and  an  additional  assumption  on  N,  then  we  may  strengthen  Theorem 
7.3.3  to  conclude  global  asymptotic  stability  as  presented  in  the  following  theorem. 

Theorem  7.3,4  Assume  all  the  conditions  of  Theorem  7.3.3  hold,  and  further  assume  that  V  <  0 
on  N ,  and  that  E  contains  only  the  single  element  q  =  0.  Then  the  closed  loop  system  defined  by 
(6.2),  (7.1),  and  (7.2)  is  globally  asymptotically  stable. 


Proof:  From  the  proofs  of  Theorems  7.3.1  and  7.3.3,  we  have  that  all  trajectories  of  the  closed 

loop  system  are  globally  bounded.  Since  the  closed  loop  system  is  autonomous  and  V  <0  globally, 
Lasalle’s  invariance  principle  [44]  implies  that  all  trajectories  converge  to  the  maximal  invariant  set 
contained  in  E,  which  by  assumption  is  the  origin,  and  thus  the  theorem  is  proved.  ■ 


We  point  out  that  the  condition  of  Theorem  7.3.3  is  not  necessary  for  global  asymptotic  stability 
of  the  origin  as  proven  in  Theorem  7.3.4,  if  global  boundedness  of  all  the  system  trajectories  may 
be  established  by  some  other  means.  It  turns  out  that  [49]  provides  two  alternative  lower  bounds 
on  the  minimum  eigenvalue  of  P  of  (7.24),  which  are 


A(P)> 


|A{(^  +  A^)Q-^}  +  ^(iA{(A  +  AT’)Q-^)y  + 
A(BP'rQ-i) 


(7.34) 


and 


A(P)> 


-W{A)  +  ja'^{A)+\{BBT^)\{Q) 

A(PB^) 


(7.35) 
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These  bounds  may  also  be  used  to  generate  sufficient  conditions  for  global  asymptotic  closed  loop 
stability  of  the  origin,  as  formalized  in  the  following  theorems. 

Theorem  7, 3.5  Assume  all  the  conditions  given  in  Theorem  7.3.1  hold  except  for  a(x)  —  0  ^  x  = 
0,  and  let  N  he  as  in  (7.32).  Also,  assume  that  on  some  set  L  C  'RT'  we  have  Q{x)  = 
where  k  >  0  is  a  real  number,  and  assume  that  LUN  —  RT'.  Additionally,  assume  that  for  all  q  E  L 
we  have  either 

fi2 [A{q)]  <  A[yl(g)  +  A^{q)]  and  \[A{q)  +  A^{q)]  >  0  (7.36) 

or 

k  >  til[A{q)]  -  (ji2[A{qMAiq)  +  A'^{q)]  (7.37) 

Then  the  closed  loop  system  defined  by  (6.2),  (7.1),  and  (7.2)  is  globally  asymptotically  stable. 

Proof:  Choose  the  globally  positive  definite,  decrescent,  and  radially  unbounded  Lyapunov 

function  (7.5).  Prom  the  proof  of  Theorem  7.3.1  we  have  V  <  0  V  g  E  iV.  Now,  for  all  q  E  L,  we 
have  Q{q)  =  kl  from  (7.22)  and  the  stated  assumptions.  Recalling  B{q)B'^[q)  —  I  and  substituting 
for  Q{q)  in  (7.34)  we  find 

A[P(g)]  >  ^A[^(9)  +  A^{q)]  +  ){\\[A(<l)AAT{q)]f+k  (7.38) 

and  it  is  easily  established  that  both  (7.36)  and  (7.37)  guarantee  that  X[P{q)]  >  ii2[A{q)],  so  that 
from  (7.19)  we  clearly  have  V  <  0  \/  q  E  L.  The  assumption  that  L  U  iV  =  then  ensures  that 
V  <  0  V  q  E  RP,  which  completes  the  proof.  ■ 

We  note  that  the  above  theorem  allows  the  nonsingularity  requirement  on  A{q)  to  be  relaxed, 
but  at  the  expense  of  having  to  choose  a  prescribed  form  of  the  state  weighting  matrix,  with  only 
the  scalar  k  left  as  a  design  parameter,  which  must  be  chosen  sufficiently  large.  The  next  theorem 
also  allows  relaxation  of  the  nonsingularity  requirement  on  A{q),  with  the  only  requirement  being 
that  Q{q)  is  chosen  ‘big  enough.’  At  this  point  it  is  interesting  to  note  that  Theorems  7.3.5  and  7.3.6 
provide  theoretical  corroboration  for  a  phenomenon  observed  in  [19],  namely,  that  state  weightings 
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must  sornetimes  be  increased  beyond  initiai  choices  to  sufficiently  large  levels  in  order  to  provide 
global  closed  loop  stability  of  an  SDRE  regulator. 

Theorem  7.3.6  Assume  all  the  conditions  given  in  Theorem  7.3.1  hold  except  for  a{x)  =  0  a;  = 
0,  and  let  N  be  as  in  (7.32).  Also^  assume  that  on  some  set  M  C'RT  we  have  for  all  q  e  M 

X[Q{q)]  >  2^A{q)]Mq)]  +  fil[A{q)]  (7.39) 

and  assume  that  MU  N  =  71^.  Then  the  closed  loop  system  defined  by  (6.2),  (7.1),  and  (7.2)  is 
globally  asymptotically  stable. 

Proof:  The  proof  is  exactly  like  that  of  Theorem  7.3.5,  except  (7.39)  substituted  into  (7.35)  (with 
B{q)B'^(q)  —  I)  guarantees  that  V  <  OV  q  e  M.  ■ 

Since  the  induced  norm  is  an  upper  bound  for  the  matrix  measure  [69],  we  have  the  following 
simplified  condition  which  can  be  used  in  place  of  (7.39). 

Theorem  7.3.7  Assume  all  the  conditions  of  Theorem  7.3.6  hold,  except  that  in  place  of  (7.39)  we 
have 

A[g(g)]  >  3a^[A{q)]  VqeM  (7.40) 

Then  the  closed  loop  system  defined  by  (6.2),  (7.1),  and  (7.2)  is  globally  asymptotically  stable. 

Proof:  The  claim  follows  trivially  from  the  proof  of  Theorem  7.3.6  and  the  fact  that  a[.4]  >  p2  (^) 
for  any  Ae  C^.  ■ 

Finally,  we  observe  that  the  above  theorems  can  be  combined  in  various  forms,  e.g.,  use  Theorems 
7.3.1,  7.3.5,  and  7.3.6  with  the  assumption  that  LUMUN  =  71^  in  place  of  the  similar  assumptions 
therein. 

7.4  Discussion 

In  this  section  we  include  some  remarks  concerning  the  full  rank,  constant  B  matrix  assumption. 
Besides  allowing  the  linear,  static  coordinate  transformation  (7.3),  and  use  of  the  bound  (7.25),  the 
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full  rank,  constant  B  matrix  assumption  serves  another  purpose.  In  Chapter  6  it  was  shown  that 
systems  of  the  form  (6.1)  which  may  be  further  written  as  (6.2),  with  full  rank  B  matrices,  are 
nonlinearly  controllable  for  all  x  in  addition  to  having  {A(x),B}  controllable  in  the  linear  sense 
for  all  X,  regardless  of  the  choice  of  A.  Thus,  the  reachable  set  from  each  x  has  dimension  n, 
so  that  the  uncontrollable  subspace  for  such  systems  consists  entirely  of  the  zero  vector.  This  is 
important  because  the  proofs  in  this  chapter  guarantee  that  the  bracketed  term  in  the  Lyapunov 
function  derivative  V  =  _  P{q)]q  is  negative  definite.  However,  -P{q)q  =  B{q)u{q)^ 

so  that  if  q  is  in  the  uncontrollable  subspace  for  the  system  (6.1),  then  q'^B{q)u{q)  =  0  and  we  are 
left  with  V  =  which  we  clearly  cannot  guarantee  to  be  negative  for  all  q  except 

under  extremely  restrictive  conditions.  The  full  rank  B  matrix  assumption  thus  guarantees  that 
q'^B{q)u{q)  =  0  only  for  ^  =  0.  Finally,  notice  that  we  may  allow  u  €  'RP' ,  where  m  >  n,  and 
Tank{B)  =  n.  In  this  case,  just  replace  B~^  with  where  (-)^  denotes  the  pseudoinverse. 

7.5  Conclusion 

We  have  given  conditions  which  guarantee  global  asymptotic  stability  of  a  class  of  nonlinear,  input- 
affine  systems  employing  SDRE  nonlinear  regulation.  The  conditions  common  to  every  theorem  are: 
full  rank,  constant  B  matrices;  constant,  positive  definite  control  penalty  matrix  R;  and  positive 
definite  state  weighting  matrix  function  Q{x)  for  all  x.  Various  additional  assumptions  on  the 
choice  of  Q  or  A  are  combined  to  obtain  the  final  sufficient  stability  conditions.  The  full  rank  B 
matrix  assumption  was  also  said  to  be  important  in  that  it  guarantees  an  uncontrollable  subspace 
consisting  of  only  the  zero  vector,  thus  validating  the  expressions  establishing  negativity  of  the 
Lyapunov  function  derivative  which  rely  on  negative  definiteness  of  an  n  x  n  matrix  at  each  point 
in  the  state  space. 
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VIII.  Lyapunov  Stability  of  Analytic  Sampled  Data  Systems  with 


Positive  Definite  Q  Matrices 

8.1  Introduction 

In  this  chapter  we  consider  sampled  data  implementation  of  the  continuous  SDRE  nonlinear  regu¬ 
lation  algorithm,  and  derive  sufficient  conditions  for  theoretically  global  asymptotic  stability  of  the 
closed  loop  system.  The  global  theoretical  result  obtained  requires  infinitely  fast  sampling,  so  that 
real  applications  will  have  bounded  domains  of  attraction,  and  thus  it  is  perhaps  more  accurate  to 
say  semiglobal  stability  for  the  sampled  data  regulator  is  proven. 

We  again  consider  the  continuous  time,  state  feedback,  input-affine,  autonomous  nonlinear  dy¬ 
namic  system  described  by 

X  =  a{x)  +  h{x)u^  a(0)  =  0 
h{x) 

,  /i(0)  =  0  (8.1) 

R{x)u 

with  state  vector  x  control  u  E  penalized  variable  z  EVf ,  and  control  penalty  matrix 

R{x)  (assumed  nonsingular  for  all  x).  We  assume  that  the  ai.,  Bij^  hi  and  Rij  are  all  real- valued 
analytic  functions  of  x  on  As  shown  in  Section  2.5,  under  the  above  assumptions  the  system 
(8.1)  can  be  written  [69]  (nonuniquely)  in  the  state-dependent  coefficient  (SDC)  form 

X  =  A{x)x  -h  B{x)u 
H{x)x 
R{x)u 

where  A  and  H  are  chosen  to  be  analytic  in  x  (see  Section  2.5.2).  We  also  assume  that  the  pairs 
{A{x),B{x)}  and  {H{x),A{x)}  are  globally  stabilizable  and  detectable,  respectively.  Furthermore, 
we  assume  that  H{x)  is  globally  nonsingular  so  that  {x)H{x)  has  full  rank  for  all  x.  Note  that 
this  assumption  on  H  actually  strengthens  the  global  detectability  assumption  on  {H{x).,A{x)} 


8-1 


to  global  observability.  Using  these  assumptions  we  will  now  establish  sufficient  conditions  under 
wliich  the  sampled  data  state  feedback  nonlinear  SDRE  regulator  yields  a  closed  loop  system  which 
is  theoretically  globally  asymptotically  stable. 

8,2  Sampled  Data  Nonlinear  SDRE  Regulation 

The  continuous  time  control  algorithm  is  given  as  before  by 

u  -  -R-\x)B'^{x)P{x)x  (8.3) 

where  JP(x)  is  the  maximal  solution  to  the  steady  state  continuous  time  state-dependent  Riccati 
equation  (SDRE) 

A^{x)P{x)  +  P{x)A{x)  -  P{x)B{x)R-^{x)B'^{x)P{x)  +  H^(x)H{x)  =  0  (8.4) 

which  is  guaranteed  to  exist  under  the  stabilizability /detectability  assumptions,  where  we  have 
defined  R{x)  =  R'^  {x)R{x)  >  0.  In  actual  implementation,  we  will  use  a  sampled  data  form  of  the 
above,  taking  measurements  (assumed  perfect)  of  the  state  variables  periodically  and  using  them  to 
generate  new  values  of  the  control.  Adopting  the  convention  x{tk)  —  Xk  and  A{x(tk))  =  Ak  (and 
likewise  for  all  other  vectors  and  matrices),  the  algorithm  is  to  apply  the  control 

Uk  =  -R^^BjPkXk  (8.5) 

at  each,  sampling  time  where  Pk  is  obtained  from  the  sampled  data  SDRE 

Al Pk  A  PkAk-  Pk  BkRZ^Bj Pk  +  Hi Hk  =  0  (8.6) 

Note  that  we  are  applying  a  constant  control  input  over  any  given  sampling  interval  [4, 4+1)5  and 
we  have  left  the  sampling  interval  size,  bk  =  4+i  “  45  unspecified  as  of  yet.  We  emphasize  that  the 
above  algorithm  is  simply  a  sampled  data  implementation  of  the  continuous  time  SDRE  regulator, 
and  not  a  discrete  time  control  law  derived  from  a  discrete  time  dynamical  system  model.  The 
motivation  for  the  above  approach  is  to  gain  a  better  understanding  of  the  continuous  controller 
through  a  discrete  time  approximation.  A  more  rigorous  discrete  time  treatment  of  the  SDRE 
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regulator  problem  is  not  pursued  in  this  dissertation,  but  is  instead  proposed  as  a  topic  for  further 
research.  To  prove  stability  of  this  algorithm,  we  shall  use  the  following  discrete  time  version  of  the 
Lyapunov  stability  theorem,  taken  from  [7]. 

Theorem  8.2.1  Consider  the  vector  difference  equation 

(8.7) 

with  transition  matrix 

^{tk-\-N,^k)  ~  JV-l^A:+yV-2  •  -  (8-8) 

Suppose  there  exist  ^  R  and  a  positive  definite  matrix  sequence  Xk  with  0  <  al  <  Xk  <  /3/  <  oo 
such  that 

f -  JYfc  =  -Nl Nk  (8.9) 

for  some  matrix  sequence  Nk,  and  all  k.  Then  (8.7)  is  stable  in  the  sense  of  Lyapunov,  and 
Vk  =  x^XkXk  is  a  Lyapunov  function  for  (8.7).  Furthermore,  if  Nk  >  0  for  all  k,  then  (8.7)  is 
asymptotically  stable.  Alternatively,  if  Nk>^  and  AVk  =  —x'^N'[NkXk  does  not  vanish  identically 
for  all  k,  any  to,  and  any  xo,  then  (8.7)  is  asymptotically  stable. 

Proof:  See  [6]  and  [36].  ■ 

8.3  Linear  Transition  Matrix  Representation 

We  first  show  that  the  closed  loop  system  using  (8.5)  can  be  written  in  the  form  of  (8.7),  provided 
6k  is  chosen  small  enough.  Using  (8.2),  for  t  £  [tk,  ifc+i)  the  closed  loop  dynamics  become 

X  =  A{x)x  -h  B[x)uk  ~  f{x,Xk)  (8.10) 

Recall  that  by  assumption  a{x)  =  A{x)x  and  B{x)  are  analytic  with  respect  to  x  on  and  so  is 
Uk  since  it  is  just  a  constant  vector.  Thus,  /  is  analytic  over  any  sampling  interval,  implying  it  has 
a  convergent  power  series  representation  in  x  and  thus  that  it  has  continuously  differentiable  partial 
derivatives.  This  last  observation  is  sufficient  to  guarantee  that  /  satisfies  a  local  Lipschitz  condition 
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for  any  Xk  6  TZ'^  [1],  so  that  (8.10)  possesses  a  unique  solution  near  Xk  which  is  continuous  and  in 
fact  analytic  in  t  if  the  system  matrices  are  analytic  in  x  [36].  Thus,  by  taking  sufficiently  close 
to  tk,  we  can  ensure  that  x  is  continuous  (and  analytic)  with  respect  to  t.  Continuity  of  x  ensures 
continuity  of  /  with  respect  to  t  over  the  sampling  interval,  and  thus  we  can  integrate  (8.10)  from 
tk  to  4+1  to  obtain 

rtk+i 


Xk+i  =  Xk+  f{x{t),Xk)dt 

Jth 

P^h+l 

■  /  A{x(t))x{t)  dt 

Jth 


fth 

-  B{x{t))  d?j  RZ^Bl 


BkXk 


(8.11) 


(8.12) 


Now  we  can  apply  the  Mean  Value  Theorem  (MVT)  [1]  for  integrals  to  each  individual  element  of 
the  right  hand  side  of  (8.11)  to  obtain 


Xi 

Xl 

X2 

- 

X2 

+  Sk 

/2(4'^) 

Xn 

fc+1 

Xji 

k 

_  ^(4”^)  _ 

(8.13) 


where  Sk  =  ^fc+i  —  4  and  G  [4,4+i]  V  ^  G  [l,n]  is  the  time  of  evaluation  of  fi  in  the  MVT. 
We  have  introduced  a  slight  abuse  of  notation  by  letting  fi  represent  the  composite  function  of  t  as 
well  as  the  function  of  x.  The  notation  on  t  is  as  follows:  the  superscripted  number  in  parenthesis 
indicates  the  row  of  /  that  is  being  evaluated  using  the  MVT,  while  the  subscript  k  simply  indicates 
association  with  the  sampling  interval  start  time  4-  Using  the  definition  of  /  we  get 


Xl 

Xl 

s,^i,-(4'V,-(4'4 

X2 

- 

X2 

“h  Sk 

Xn 

k+1 

Xn 

k 

-SkBkRf.  BlPkXk 


(8.14) 
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where  we  have  defined 


Bk  = 


Sii(4'^)  Si2(4'4 

52i(4'4  522(4'4 


•S2m(4 


(8.15) 


5„i(4"4  5„2(4"^)  ••• 

Consider  now  only  the  second  term  on  the  right  hand  side  of  (8.14).  This  term  may  be  decomposed 
into  a  sum  of  n  terms  as 

''^u(4'4  ^12(4'4  •••  ^ln(4'^) 

0  0  0 


h 


+ 


+  ^k 


A2i(4'^)  2422(4'') 


(2)x 


^^(4^4 


»;(4^4  + 


0 


0 


_a„i(4"4  ^„2(4"4  •••  ^n44”^) 

=  +  ^i^^a;(4^4  +  •  •  •  +  ^i"^®(4"^)] 


^(4"^) 


(8.16) 


(8.17) 


Note  that  is  a  matrix  with  its  ith  row  equal  to  the  zth  row  of  A  evaluated  at  and  zero 
elsewhere.  Now,  by  taking  advantage  of  the  fact  that  each  of  the  above  may  in  turn  be  written 

as 

^(4*^)  =  ^k+  f{x{t),Xk)  dt  (8.18) 

Jtk 


(0 
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and  again  applying  the  MVT  to  the  integrals  in  (8.18),  we  see  that  (8.17)  is  equal  to 


\ 

thA^^ 

i=zl 

xk  +4*^ 

-6^^^B<^^R;^BlPkXk 

S,-^ni(4“V.(4“^) 

/ 

(8.19) 


where  <5^^^  =  4^^ 


5ii(4*'’) 

^12(4''^) 

••• 

52i(4*"’) 

•••  S2„(4^4 

■Bnl(4‘”') 

•••  e„„,(4"4 

(8.20) 


The  notation  4*'^^  G  indicates  this  time  is  generated  from  the  ith  previous  value  of  tu, 

and  that  the  jth  row  of  f  is  being  evaluated  using  the  MVT.  The  subscript  k  is  again  to  indicate 
dependence  on  the  original  starting  time.  Note  we  have  4*^  <  ^a+i  V  f  so  that  all  4*^  <  ^fc-  We 
can  continue  with  another  application  of  the  MVT,  noting  that  each  application  generates  n  new 
t  values,  and  we  adapt  our  notation  by  adding  an  additional  superscript  to  tk,  Sk,  Ak  and  Bk-  By 
defining 


i(s)  _ 


and  defining  for  (8.19)  analogously  to  how  was  defined  in  (8.17),  and  subsequently  setting 


E^: 

i=l 

(i) 


(i) 

k 


(8.21) 


4®  =  E4“. 

3=1 


(8.22) 


then  (8.19)  becomes 


4 


i=l 


i=l 


Xk+0(S^,A^)x(ti^^‘^) 


(8.23) 


where  the  O  notation  indicates  terms  of  order  3  in  both  S  and  A  premultiplying  the  terms  in  x, 
and  I  is  another  integer  index  ranging  from  1  to  n.  Now,  (8.18)  and  the  MVT  may  be  repeatedly 


applied  to  the  of  (8.23),  leading  to  an  infinite  matrix  series  expression  in  6  and  which,  if 
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convergent,  implies  that  the  higher-order  terms  contribute  nothing  (the  higher-order  terms  are  the 
limit  of  the  sequence  of  6 A  products  as  the  number  of  terms  tends  to  infinity,  which  must  be  zero 
for  a  convergent  series).  Furthermore,  each  time  the  MVT  is  applied  to  an  expression  of  the  form 
of  (8.18),  it  introduces  another  term  with  a  single  postmultiplier  in  it  -  thus,  another  infinite 
series.  Assuming  both  these  series  converge,  then  (8.11)  may  be  written 


= 


-6^ 


i=l 

^(0  p(*) 

i=l 


Xk 


^k  Pk^k 


(8.24) 


This  yields  a  transition  matrix  representation,  as  formally  stated  and  proven  in  the  following  theorem. 


Theorem  8.3.1  Consider  the  continuous  time  open  loop  system  (8.2)  with  discrete  time  control  law 
given  by  (8.5)  and  (8.6)^  where  A{x),B{x),H{x)  and  R{x)  are  analytic  in  x  £  TZ^j  and  that  the 
pairs  {A(x)^B{x)}  and  {H (x),  A{x)}  are  stabilizable  and  detectable,  respectively,  for  all  x.  Define 


Bk{A)=:  max  |Aij(i)| 

Then  if  the  sampling  interval  size  Sk  =  tk+i  —  tk  is  chosen  such  that 

,  ^  1 
'  ^  nBk{A) 

the  closed  loop  system  (8.10)  can  be  written  in  the  form 

Xk+l  =  ^kXk 


(8.25) 


(8.26) 


(8.27) 


where 


h 


i-1 


Rl^BlPu 


(8.28) 


L  i=i  J 

Proof:  By  the  development  involving  the  MVT  in  (8.11)  -  (8.24),  we  must  simply  prove  that 

both  infinite  matrix  series  are  convergent  for  (8.28)  to  be  valid.  Since  a  matrix  series  converges  iff 
each  individual  matrix  element  series  converges  [42],  we  need  to  prove  element-wise  convergence. 
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Recall  that  the  Aij  and  Bij  are  continuous  functions  of  x,  and  we  have  restricted  4+i  so  that  x  is 
a  continuous  function  of  t  on  the  closed,  bounded  interval  [i^:,  ifc+i]-  Thus,  the  Aij  and  Bij^  and  in 
fact  the  absolute  values  \Aij\  and  \Bij\^  are  continuous  functions  of  t  on  a  compact  subset  of  7i,  so 
that  each  \  Aij\  and  \Bij\  attains  a  maximum  over  Let  Bk{A)  be  defined  as  in  (8.25)  and 

Bk{B)  =  max  \Bij{t)\  (8.29) 

FYoin  (8.28),  define 

=  I +  +Skf^  4'^  +  •  •  •  (8-30) 

4=1 

Tk  =  SkBk  +  6k  +  •  •  •  (8.31) 

4=1 

with  the  higher-order  terms  determined  by  repeated  application  of  the  MVT  as  described  above. 
Then  from  the  definitions  in  (8.25)  and  (8.29)  and  the  fact  that  we  have 

I  <  1  +  6kBk{A)  +  6lnBl(A)  +  SWbUA)  +  ••■ 

=  1  -  ^  ^  I I  (8-32) 


\Tk,,\<6kBk{B)  +  6lnBk{A)Bk{B)+8WBl{A)Bk{B)+--- 

=  6k  |^[«ifenSfc(^)]’-|  Bk{B)  (8.33) 

Therefore,  both  \  ^kij\  and  IF^.J  will  be  convergent  so  long  as  the  geometric  series 

CXD 

Y^[6knBk(A)Y  (8.34) 

r=0 

converges.  It  is  well  known  that  a  sufficient  condition  for  convergence  of  (8.34)  is  to  require 


\6knBk{A)\  <  1 


(8.35) 


Since  absolute  convergence  of  an  infinite  series  implies  convergence  of  the  series  [1],  we  may  ensure 
validity  of  (8.28)  by  choosing  Sk  according  to  (8.26). 
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We  have  thus  proven  that  our  system  dynamics  can  be  written  in  the  form  of  (8.27),  provided 
each  4  is  chosen  small  enough.  Note  that  if  4  is  chosen  small  enough  to  guarantee  validity  of  (8.28) 
up  to  the  next  sampling  time  ,  then  clearly  we  also  have  such  a  valid  expression  for  any  x  within 
the  sampling  interval,  i.e.,  any  such  that  6  ■ 

8.4  Stability  Via  Lyapunov  Theory 

We  shall  now  make  use  of  Theorem  8,2.1  to  prove  stability  of  the  sampled  data  discrete  time  system, 
which  we  now  write  via  Theorem  8.3.1  as 

XH+i  =  ^kXk  =  [/  +  4(4''^  -  BkRl^BlPk)  +  0{6\A^)  +  0{6\AB)]xk  (8.36) 

where  the  last  two  terms  in  the  bracketed  expression  represent  higher-order  terms  in  the  infinite 
series  expansion  of  We  first  consider  only  variations  over  the  entire  sampling  interval,  and  then 
address  intersample  behavior.  Define 


=  Af^  -  Ak 

(8.37) 

ABk  =  Bk  —  Bk 

(8.38) 

Then  (8.36)  may  be  written 

=  [/-}-  4(^fc  ”  BkFik  ^ ^B^Pk) 

+  0(S\A^)  -f  0(S\AB)]xk  (8.39) 

Now,  recognizing  that  A^  ~  Bk  Bj^  Pk  is  just  the  closed  loop  continuous  dynamics  matrix  sampled 

at  time  4?  we  define  this  matrix  as  Fk  to  get 

Xk+i  =  [I  +  6kFk  +  -  ABkR^^BjPk)  +  0(6^,  A^)  +  Oi6^,AB)]xk  (8.40) 

Recalling  Theorem  8.2.1,  we  seek  to  prove  that  there  exists  a  bounded  positive  definite  matrix 
sequence  Xk  such  that  the  Lyapunov  function  14  =  ^J^k^k  has  a  negative  variation  A14  along  the 
trajectory  for  all  k.  Using  (8.40)  we  see  that 
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A14  =  ^fc+1  - 

—  ^kl^k^k+l^k  -  Xk]Xk 

=  xj  Xjt-i-i  —  Xk  +6k  Xk-ki  +  Xk+iFk  +  AAjXk+i  +  Xk-\-iXAk 

-  {ABkRl^BlPkfXk+x-Xk+i{ABkRk^BlPk)^ 

+  Xk  (8.41) 

where  m  =  {0, 1, 2}  and  the  O  notation  represents  terms  of  order  2  or  higher  in  6,  which  contain  A 
and  B  terms  of  a  combined  order  no  greater  than  the  order  of  6,  We  now  prove  that  under  certain 
conditions  there  always  exists  such  a  sequence  X*.  The  proof  is  constructive  by  nature;  i.e.,  we 
show  how  such  a  sequence  may  be  constructed.  Before  proceeding  we  make  a  crucial  observation 
concerning  the  nature  of  the  solution  to  the  continuous  time  SDRE  (8.4).  It  has  been  proven  [7]  that, 
if  the  elements  of  the  matrices  H,  H  involved  in  a  continuous  time  algebraic  Riccati  equation 

depend  continuously  on  some  parameter  and  if  stabilizability  and  detectability  assumptions  hold, 
then  the  maximal  solution  P  also  depends  continuously  on  that  parameter.  In  fact,  it  has  been  shown 
[58]  that  the  same  may  be  said  for  analyticity  with  respect  to  a  parameter  under  the  assumptions 
of  analyticity  of  the  matrix  elements  and  stabilizability  and  detectability. 

Theorem  8.4*1  Assume  the  conditions  in  Theorem  8.3.1  holdj  and  further  assume  that  H{x)  is 
globally  nonsingular  so  that  >  0  for  all  x,  and  that  the  solution  to  the  SDRE,  Pkf  is 

such  that  limj(;^oo  Dk  exists.  Then  for  6k  sufficiently  small,  the  conditions  of  Theorem  8.2.1  hold 
and  the  closed  loop  system  (8.10)  is  globally  asymptotically  stable. 

Proof:  First,  note  that  the  SDRE  solution  P{x)  is  positive  definite  for  any  x,  since  {H{x),A{x)} 
is  observable  under  our  assumption  of  global  iionsingularity  of  H{x)  [76].  For  any  U  G 


8-*  10 


define 


A, 


X}z 


{if  Xk  ^  0) 


=  0  {if  Xk  =  0) 


(8.42) 


where  8r~tr  —  tk  and  let 


(8.43) 


where  Pr  is  the  SDRE  solution  at  Xr,  and  5^  is  a  positive  scalar  defined  by  the  recursive  expression 


r**'« 

=  Sk  {otherwise) 


(8.44) 


with  So  =  1  and  where  e  is  an  arbitrarily  small  positive  tolerance  selected  so  that 


e  <  mm  [X{HjHk)] 


(8.45) 


where  A  is  the  minimum  eigenvalue.  Using  (8.43),  (8.41)  becomes 


—  X^[S}i^iPk^l  SkPk]Xk 

H-  ^k+l^k^k  d-Pfc+li^fc  -j-  AAjPk+i  -fP/s+iAylfc 

-  {ABkR^^B][Pf,)'^Pk^i  -Pk^iiABkRj^^BjPk)  Xk 
-\-xlO{6^,A^-^B^)xk 


(8.46) 


Notice  that  if  >  0  in  (8.9),  then  (8.9)  is  equivalent  to  (8.41)  being  negative  for  all  Xk  other  than 
zero,  and  thus  we  want  (8.46)  to  be  negative  for  all  Xk  other  than  zero.  Prom  (8.42)  and  (8.44)  we 
have 


xl[sk+iPk+i  -  SkPk]xk  <  €Sk+iSkxJxk  V  k  (8.47) 

so  that  the  first  term  in  (8.46)  is  always  bounded  by  an  arbitrarily  small  positive  constant.  We  now 
make  use  of  the  analyticity  of  the  Pij,  Aij  and  Bij  with  respect  to  t  to  expand  in  a  neighborhood 
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of  tk  and  write 


p..  -p,. 


dt  '  “*■ 

I 

ifc 


(Sr  +  •  •  •  =  Pij^  +  Pijjr  +  ■ 


A-.  -A- 


■  dt 

D  __  D  , 

-^ijr  —  ^ijk 


<5p  +  *  •  *  —  Aij^  -\-  A^^dr  +  ■ 


tk 


<^r  H - ”  Bij^  +  ^ijk^’^  “*■ 


tk 


(8.48) 

(8.49) 

(8.50) 


for  tr  €  [iAs)  4+i]5  where  and  B^,  are  the  matrices  of  partials  of  P,  A,  and  B  with  respect  to 

t  evaluated  at  tk,  respectively.  Using  (8.37),  (8.38),  and  (8.48),  (8.46)  becomes 


^Vk  <Sk-\^iSkxl 


el  +  {Pk  A  PlSk)  +  {Pk  +  Plh)Fk 


+  (If)  -  AkfiPk  +  P'k6k)  +  {Pk  +  P'kSkKAi^^  -  Ak) 

-  ((Bfc  -  Bk)R^^BlPk'^  {Pk  +  P'kSk) 

-  {Pk  +  P'kSk)(^{Bk  -  Bk)R^^BlPk'^yk+xlO{S-^)xk 


(8.51) 


where  the  higher-order  terms  from  (8.48)  have  been  absorbed  into  the  O(S^)  term  (note  -  there  are 
still  no  ^  or  P  terms  with  combined  order  greater  than  the  order  of  6  in  this  term).  Prom  (8.16), 
(8.17),  (8.21),  and  (8.49)  we  have 


A2l(if)  ^22(tf)  •••  A2n(if) 


^nl(4"’)  ^n2(4"^)  •••  ^n„(4”^) 


^11*  +  iSf  ^iifc  H - 

^21k  +  ^21*  +  •  •  • 


"^12*  +  ••• 
^22fc  +  <Sf  ^22*  H - 

An2k  +  'SF'^7i2fc  +  •  •  ■ 


Aink  +  (Sf  H - 

+  (Sf  H - 

A  A^  A* 

^Tinjt  ^  ^k  -^nrik  ' 


—  Ak  +  Asj^Af.  +  0(6^^) 


(8.52) 
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where 


=  (8-53) 

SO  that 

Af^  -Ak  =  ^sA+0{6‘^)  (8.54) 

Similarly,  from  the  definition  of  Bk  and  (8.50),  we  have 

+  (8.55) 

Now  (8.51)  may  be  written 

^Vk  <  Sk^iSk^x^  el F^Pk -\- PkFk -\-Sk{F^Pk  +  Pk^k) 

+  (■^k)'^ ^^kPk Pk^Sk^k  ^k 

-  xl  (^As,B^R-,^BlP^  Pk  +  Pk  iksuB'kRk^BlP^  ]  xu  | 

+  xlO{S^)xk  (8.56) 

Using  the  fact  that,  for  symmetric  M, 

x^Mx  <  a(M)x'^x  (8.57) 

and 

—  x'^ Mx  <  “  a{M)x^x  (8.58) 

where  a  and  a  denote  the  maximum  and  minimum  singular  values,  respectively,  and  that 

x'^A^^x  <  SkX^x  (8.59) 


Equation  (8.56)  becomes 

AVfc  <  Sk+iSkxllel  +  Fl Pk  +  PkFk]xk  +  Sk+i6lxl[F^Pl  +  P'kFkjxk 
+  Sk+i6la  {A'k)'^Pk+PkA'k  x^Xk 

-Sk+iSls.  (^B'kRk^BlPk^  Pk  +  Pk^B'kB-l’^BlP^  xjxk 
+  xlOi6^)xk  (8.60) 

=  Sk+i6kxl  el  +  F[ Pk  +  PkFk  Xk  +  xj Oi6^)xk  (8.61) 


where  the  last  four  terms  in  (8.60)  have  been  combined  into  the  O  term  in  (8.61),  which  is  allowable 
since  P^,  and  are  guaranteed  finite  by  our  analyticity  assumption.  Now,  it  is  easy  to  show 
that  the  SDRE  (8.6)  can  be  rearranged  to  give 

F^Pk  +PkFk  =  -Hi Hk  -  PkBkRk^ Bj P^  =  -Qk  (8.62) 


which  is  negative  definite  since  HjHk  >  0  was  assumed.  Thus,  Qa;  >  0  and  (8.61)  becomes 


AT4  < 


Xk  -i-  xlO{S^)xk 


(8.63) 


and  from  (8.45)  we  are  guaranteed  that  el  -  Qk  <0  for  any  fc.  Now,  there  exists  a  4  sufficiently 
small  so  that  the  first  term  in  (8.63)  dominates  and  AT4  <  0-  Note  that  if  6k  is  chosen  to  ensure  the 
right  hand  side  of  (8.63)  is  negative,  then  for  any  smaller  6k  we  still  have  AV^  <  0  since  Qk  remains 
constant  and  the  possibly  destabilizing  contributions  due  to  P^,  and  Bf.  become  smaller.  Thus, 
AVk  <  0  for  all  tr  €  (4 5  so  that  the  intersample  dynamics  are  well  behaved. 

Finally,  we  need  to  show  there  exist  positive,  real  over-  and  underbounds  for  Xk,  as  defined  by 
(8.43).  From  the  assumption  that  lim^^oo  Pfe  exists,  we  have  that  ||Pa;||2  is  bounded  above,  while 
from  the  facts  that  P*  is  positive  definite  and  analytic,  we  are  guaranteed  ||Pa;||2  is  bounded  below 
away  from  0.  Now,  for  convergent  P^,  (8.44)  guarantees  that  Sk  is  the  product  of  a  finite  (but 
possibly  very  large)  number  of  positive  real  numbers  for  all  k.  Thus,  the  set  of  all  Sk  is  a  finite  set 
of  positive  numbers  which  is  guranteed  to  possess  maximum  and  minimum  elements,  both  of  which 
must  be  positive.  The  products  of  the  lower  and  upper  bounds  on  Sk  and  ||Pa;||2,  respectively,  give 
a  and  /3  in  Theorem  8.2.1.  ■ 


8.5  Discussion  of  Sampling  Interval  Size 

With  regard  to  practical  application,  (8.60)  and  (8.62)  clearly  show  the  interplay  between  the  sample 
period  size  6k,  the  state  weighting  matrix  Q  =  the  Riccati  solution  P,  and  the  system 

dynamics  parameters  A  and  B.  The  interpretation  is  that  the  Q  matrix  needs  to  be  selected  ^big 
enough’  and  the  sampling  interval  ‘small  enough’  so  that  linear  terms  in  the  Taylor  series  expansions 
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of  Xk  dominate,  and  that  additionally  the  rates  of  change  of  P,  A,  and  B  are  small  enough  to  be 
dominated  by  the  stabilizing  effects  of  Q.  It  is  interesting  to  note  that  this  heuristic  explanation 
of  the  theory  agrees  completely  with  the  observations  in  [19]  and  the  analytical  results  found  for 
the  case  of  nonlinear  systems  written  as  (8.2)  in  [59},  in  which  conditions  guaranteeing  stability  of 
quasi-linear  parameter-varying  systems  are  derived.  It  can  be  deduced  from  (8.4)  that  the  rate  of 
change  of  P  will  be  constrained  by  the  rates  of  change  of  J5,  jR,  and  Q,  so  that  the  system 
dynamics  are  the  determining  factor  if  Q  and  R  are  chosen  constant.  For  highly  nonlinear  systems 
(ones  with  high  powers  of  x  in  the  Taylor  series  expansions),  we  would  expect  large  instantaneous 
time  rates  of  change  far  from  the  origin  due  to  the  contribution  from  the  partial  with  respect  to  x. 
For  fixed  Q  and  achievable  sampling  rate,  by  the  above  proof  we  would  clearly  expect  such  systems 
to  have  smaller  domains  of  attraction  than  less  highly  nonlinear  systems.  Hence  the  claim  of  global 
stability  for  the  nonlinear  SDRE  regulator  would  not  be  true  for  systems  sampled  at  a  finite  rate, 
in  general. 

When  actually  selecting  sampling  intervals,  several  factors  play  a  part.  Recall  from  Theorem 
8.3.1  that  6k  must  be  selected  small  enough  to  make  the  state  transition  matrix  representation  valid, 
and  we  had  (8.26)  as  a  requirement  for  convergence  of  the  infinite  series  representation.  Additionally, 
from  considering  the  stability  proof  in  Theorem  8.4.1,  we  see  that  we  need  nSk^  and  nSkAkSk  to  be 
very  small  compared  to  Ak  in  order  to  ensure  that  the  first-order  terms  dominate  the  state  transition 
matrix  expression.  Also,  we  need  6k  small  enough  to  ensure  that  the  Taylor  series  expansions  of 
the  system  parameters  are  valid  in  a  neighborhood  of  the  sampling  times,  and  moreover,  that  the 
constant  and  linear  terms  in  these  expansions  dominate.  Finally,  the  changes  in  P,  A,  and  B  between 
sampling  times  must  be  very  small  compared  to  Q  at  the  given  sampling  time.  Although  several 
constraints  exist,  they  all  require  6k  to  be  ‘small’,  and  thus  are  not  disjoint  but  rather  overlapping 
constraints  which  may  be  satisfied  by  choosing  6k  small  enough. 
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8.6  Conclusion 


To  summarize  our  results,  we  have  shown  that  if  lim^_>oo  Pk  exists,  and  if  the  matrices  A,  J?,  R 
and  H  meet  the  following  assumptions: 

•  analyticity  in  x 

•  {A{x),B{x)}  stabilizable  for  all  x 

•  H^{x)H{x)  >  0  for  all  x 

then  there  exists  a  set  of  sampling  times  4  such  that  the  closed  loop  system  using  the  sampled  data 
state  feedback  SDRE  regulator  is  asymptotically  stable  starting  from  any  finite  initial  condition. 
From  a  physical  point  of  view,  stability  will  be  enhanced  by  choosing  larger  state  weighting  matrices, 
at  the  expected  price  of  higher  control  usage.  For  highly  nonlinear  systems,  realistic  constraints  on 
sampling  rates  and  control  authority  will  severely  limit  domains  of  attraction. 
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IX.  Lyapunov  Stability  of  Analytic  Sampled  Data  Systems  with 


Positive  Semidefinite  Q  Matrices 


9.1  Introduction 

In  Chapter  8  it  was  shown  that  if  the  sampled  data  state  feedback  nonlinear  regulation  state- 
dependent  Riccati  equation  (SDRE)  yields  a  convergent  solution,  then,  under  some  additional  as¬ 
sumptions,  the  closed  loop  system  will  be  semiglobally  asymptotically  stable.  In  this  section  we 
review  the  sampled  data  SDRE  control  algorithm  and  the  assumptions  used  to  prove  the  stability 
claim.  In  the  remainder  of  the  chapter  we  then  show  how  the  restrictive  assumption  of  Q  >  0  for 
all  X  can  be  relaxed  to  Q  >  0  for  all  x,  thereby  greatly  increasing  both  design  flexibility  and  the  set 
of  systems  for  which  stability  may  be  proven. 

We  again  consider  regulation  of  continuous  time,  state  feedback,  input-affine,  autonomous  non¬ 
linear  dynamic  systems  of  the  form  (8.1)  with  state  vector  x  e  control  vector  u  e  penalized 
variable  z  and  control  penalty  matrix  R{x)  (assumed  nonsingular  for  all  x).  We  assume  that 

a  and  h  are  real-valued  analytic  functions  of  x  on  We  thus  can  again  write  (8.1)  (nonuniquely) 
in  state-dependent  coefficient  (SDC)  form  (8.2)  where  A  and  E  are  chosen  to  be  analytic  functions 
of  X,  The  continuous  control  algorithm  is  again  given  by  (8.3)  and  (8.4),  and  once  more  we  assume 
that  the  pairs  {A{x),B{x)}  and  {H{x),A{x)}  are  respectively  stabilizable  and  detectable  for  all  x 
to  ensure  global  solutions  of  (8.4)  and  (8.6)  exist.  Upon  substituting  the  control  (8.3)  into  (8.1)  we 
obtain  the  closed  loop  dynamics 

X  =  [A{x)  -  B{x)R-^  {x)B'^{x)P{x)]x  =  F{x)x  (9.1) 

where  F{x)  is  guaranteed  to  be  a  Hurwitz  matrix  for  all  x.  When  actually  implementing  the  above 
algorithm,  we  will  use  the  sampled  data  control  algorithm  given  by  (8.5)  and  (8.6),  which  assumes 
perfect  measurements  of  the  state  vector  x  and  new  control  values  at  sampling  times  4-  We  thus 
again  have  a  control  which  is  held  constant  over  the  sampling  interval  [45  4+i)- 
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9,2  Review  of  Stability  Proof  for  Q{x)  >  0 


In  Chapter  8  it  is  proven  that  under  the  discrete  time  control  sequence  (8.5),  the  closed  loop  dynamics 
admit  a  difference  equation  formulation 

Xr  —  ^krXk  V  ^7-  E  (^*^) 

provided  the  sampling  intervals  6k  =  iit+i  -  4  are  kept  small  enough,  where  ^kr  is  an  infinite 
matrix  series  which  may  be  approximated  by  truncating  higher-order  terms.  In  order  to  prove 
global  asymptotic  stability  of  the  closed  loop  system,  it  is  assumed  that  Pk  converges  to  some 
matrix  limit  jP,  and  H(x)  is  assumed  to  be  nonsingular  for  all  x  so  that  Qk  =  H^Hk  >  e/  >  0  V  A:. 
This  assumption  on  H  strengthens  the  global  detectability  assumption  on  {H{x)jA{x)}  to  global 
observability,  so  that  the  SDRE  solution  >  0  V  A:  [76],  and  thus  we  have  P  >  0.  A  positive  definite 
Lyapunov  function  for  the  system  is  then  defined  for  any  4  €  (4j^fc+i]  as  (recalling  x{tr)  =  Xr)) 

V{Xr)  =  SrXjPrXr  (9.3) 


where  Pr  is  the  SDRE  solution  at  Xr^  Sr  is  a  positive  scalar  defined  by  the  recursive  expression 

xJPkXk 


Sr  =  ■  gfc  {if  X  >  e) 

Xf^  ±rXk 

=  Sk  (otherwise) 


(9.4) 


with  So  =  1  and  where  e  is  an  arbitrarily  small  positive  tolerance  selected  so  that 


€  <  mm  [A(PjPfc)] 


(9.5) 


where  A  denotes  the  minimum  eigenvalue,  and  where 

r.T 


SrpXf^  Xk 

=  0  {if  Xk  =  0) 


(9.6) 


for  =  tr  —  tk-  It  is  then  shown  that  the  Lyapunov  function  variation  over  the  interval  from  tk  to 


AVkr  =  Vr-Vk 
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=  xJiSrPr  -  SkPk)Xk  +  SrSrxJlF^ Pk  +  PkFk]Xk  +  xlO{6l)Xk 
<  -SrSrxl[-eri-Qk+PkBkRk^BlPk]xk+xlO{6l)xk  (9.7) 

is  guaranteed  negative  for  all  tr  €  {tk,tk+i],  and  for  all  k,  provided  the  system  matrices  are  analytic 
in  X  and  6k  is  selected  small  enough.  Note  that  the  final  term  in  (9.7)  stands  for  higher-order 
terms  in  6k^  Furthermore,  convergence  of  guarantees  boundedness  of  Pk  and  Sk  from  above  and 
below  (away  from  zero),  so  that  (9.3)  is  also  radially  unbounded,  thus  allowing  the  claim  of  global 
asymptotic  stability  by  standard  Lyapunov  type  arguments. 

9.3  Stability  for  Q{x)  >  0 

9. 3 A  LaSalle^ s  Invariance  Principle  and  Redefined  Lyapunov  Function 

The  global  nonsingularity  assumption  on  H (x)  is  restrictive,  and  we  thus  seek  means  of  relaxing  it 
by  allowing  AVkr  ^  0  and  invoking  LaSalle’s  Invariance  Principle  [44,  69]  to  maintain  the  stability 
claim.  For  ease  of  reference  we  thus  present  a  discrete  time  version  of  this  theorem,  taken  from 
[20].  A.  prerequisite  is  the  definition  of  an  invariant  set  A  set  M  C  TiP'  is  said  to  be  invariant  with 
respect  to  (9.2)  if,  for  some  to  >  0, 

a?(io)  =  xo  e  M  ^  x{tk)  =  Xk  e  M  y  tk  e  R-{-  (9.8) 

where  Xk  represents  the  solution  of  (9.2)  at  time  tk  starting  from  xq  at  time  to.  We  now  give  the 
theorem. 

Theorem  9.3.1  (Discrete  Lasalle  Invariance)  Let  be  an  invariant  set  of  (9.2),  and  let  V  : 
n  — ^  71^  be  a  continuous  function  V{x)  such  that  AT4  =  —  T4  <  0  V  E  Also,  let 

E  —  {xk  G  ft  I  AVk  —  0},  and  let  M  be  the  maximal  invariant  set  contained  in  E,  i.e.,  the  union  of 
invariant  sets  contained  in  E.  Then  every  bounded  solution  x{tk)  starting  in  ft  converges  to  M  as 
tk  oc. 

Proof:  See  [20].  ■ 
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It  is  well  known  [11,  69]  that  Theorem  9.3T  offers  increased  flexibility  in  proving  stability  of  au¬ 
tonomous  systems,  because  it  does  not  require  V  to  be  strcitly  positive  definite,  nor  does  it  require 
Ay  to  be  strictly  negative,  and  it  also  relaxes  the  assumption  on  V  to  continuity  only.  This 
freedom  comes  at  the  expense  of  somehow  establishing  boundedness  of  system  trajectories,  however. 
In  our  case  we  have  boundedness  by  virtue  of  the  fact  that,  since  F  is  a  continuous  (analytic)  func¬ 
tion  of  X,  convergence  of  Pk  implies  convergence  of  x*,  which  implies  boundedness  of  Xk  [1].  Also 
note  that  under  our  assumption  of  sufficiently  small  6r^  the  analyticity  of  P  may  be  used  to  write 
Pr  ^  Pk-\-  SrPk  where  Pj,  is  the  matrix  of  time  derivatives  of  F  evaluated  at  4,  so  that  for  Xk  ^0 
(9.6)  can  be  replaced  by 

\/  tr  e  [tk.tk^l)  (9.9) 

xlxk 

Thus,  for  sufficiently  fast  sampling,  A^  is  the  same  for  all  values  of  time  in  the  sampling  interval, 
so  that  only  one  comparison  versus  e  need  be  made  per  sampling  interval,  and  thus  either  Sr  =  Sk 
for  all  tr  £  [4,4+1)  or  Sr  is  determined  from  the  first  line  of  (9.4)  for  all  U  £  [4,4+i)-  In  the 
first  case,  s  is  constant  and  thus  continuous  over  the  sampling  interval,  while  in  the  second  case  the 
continuity  of  F  gives  s  as  a  continuous  function  of  x.  Thus,  V  is  continuous  as  required  for  use  in 
Theorem  9.3.1.  However,  with  positive  semidefinite  Q  —  we  cannot  guarantee  satisfaction  of 

(9.5),  so  that  the  definition  of  the  Lyapunov  function  must  be  altered  to  guarantee  negativity  of  its 
variation,  or  establish  stability  via  Lasalle’s  Invariance  Principle.  We  therefore  maintain  (9.3),  but 
now  we  define  Sr  for  4  €  (tft,  4+i]  according  to 

5,  =  [if  xlP'kXk  >  xl[Q,  +  PkBkRk^BlPk\xk)  (9.10) 

=  Sk  (otherwise)  (9.11) 

where  PI,  is  as  above  and  again  we  take  Sq  ™  1.  By  arguments  analagous  to  the  above,  such  a 
definition  gives  a  continuous  V(x),  and  has  the  result  of  changing  the  Lyapunov  function  variation 
to 

AFfc,  =  -SrSrxllQk  +  PhBkR^^BlPk]xk  +  xlO{Sl)xk  (9.12) 
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for  (9.10)  so  that  the  el  term  in  (9.7)  is  no  longer  present.  For  (9.11)  we  get  a  resulting  AVkr  as 
given  in  (9.12),  plus  the  possible  addition  of  an  uncanceled  nonzero  Srxl{Pr  -  Pk)^k  term,  whose 
contribution  to  AVkr  is  not  large  enough  to  ruin  its  negativity.  In  simple  terms,  we  leave  the  x^Px 
term  alone  when  the  first-order  terms  produce  a  negative  AF,  and  we  introduce  a  scaling  to  eliminate 
the  growth  of  x'^Px  when  it  is  sufficiently  large  to  be  detrimental  to  stability. 

9,3,2  Characterization  of  E 

We  thus  now  have  a  continuous  Lyapunov  function  with  nonpositive  variation  so  that  we  are  moti¬ 
vated  to  study  invariant  sets  of  (9.2)  such  that  for  some  E  the  necessary  condition 

SrxJtlQk'  Pk'Bk>RkfB'^,Pkf]xk>  -  0  (9.13) 

holds.  To  incorporate  the  notion  of  invariance  and  strengthen  (9.13)  to  a  necessary  and  sufficient 
condition,  we  need  to  identify  when  AV^r  =  0  V  ir  6  (4j^fc+i]  sind  for  all  k  >  k'.  Thus,  recall  that 

An,  -  n  -  n 

=  S^Xj,  P-pXr  ^kXf,  PkXk  (^*^'^) 

“  ^k  r^kr  ^kPk]Xk  (9.15) 

where  ^kr  is  the  state  transition  matrix  from  Xk  to  Xr  as  defined  in  (9.2),  so  that  from  (9.14)  we 
have  the  four  possibilities 

i.  An,  =  0  if  a; A:  €  Af{Pk}  and  x,  E  A7{P,}  where  Af  represents  the  null  space  of  a  matrix, 
while  from  (9.15)  and  (9.11)  we  have 

ii.  Anr  =  0  if  =  / 

iii.  A,T^,  —  0  if  x^p  ^  Xf,  and  SpXj,  PpXp  —  s^x^  P^x^  —  c  0,  and 

iv.  AVkr  ”  0  if  =  0 

at  least  one  of  which  must  be  satisfied  for  all  t,  €  and  all  k  >  k'  for  invariance  to  hold. 

Thus,  items  i,  ii,  iii,  and  iv  above  give  a  complete  characterization  of  the  set  E  in  Theorem  9.3.1. 
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Now,  considering  item  ii,  from  (9.1)  we  have  for  sufficiently  small  6r  that  ^kr  =  /  iff  F{xr)xr  — 
0\/  tr  £  [tkjtk-\-i]i  while  item  i  above  implies  that  PrXr  =  xj Pr  —  0  for  all  tr  in  the  interval.  Item 
hi  requires  for  all  4  €  [tk^  that  FrXr  7^  0,  PrXr  ^  0,  and  the  trajectory  evolves  in  such  a  way 
that  SrxJPrXr  remains  constant  and  nonzero.  For  sufficiently  small  Sr,  it  is  easily  shown  that  this 
nonzero  rate  of  change  condition  on  the  quadratic  form  is  equivalent  to  orthogonality  of  the  two 
nonzero  vectors  Fx  and  Px,  Thus,  by  considering  the  set 

E  -  {xk  e  x^PkFkXk  =  0  or  Sk  -  0}  (9.16) 

we  have  an  alternate  but  completely  equivalent  characterization  of  E.  Now,  since  x^PFx  = 
x^F^Px  we  may  consider 

E  =  {xke  7^^|  xl[F^Pk  +  PkFk]x  =  0orsk-=:0}  (9.17) 

the  first  part  of  which  is  well  known  to  be  equivalent  to  the  set  of  all  Xk  such  that  (9.13)  holds  (with 
the  appropriate  change  in  subscripts),  by  simple  manipulation  of  the  SDRE  (8.4).  Thus,  we  seek  to 
identify  all  invariant  sets  of  (9.1)  such  that 

E^{xe  PP\  x'^lQ  +  PBR-^B'^P]x  =  0  or  s{x)  -  0}  (9.18) 

and  guarantee  asymptotic  stability  to  the  origin  of  all  trajectories  contained  in  these  invariant  sets. 

9,3,3  Nonlinear  Observability  and  Invariant  Sets 

Let  us  consider  the  first  expression  in  (9.18),  namely 

x'^lQ  -h  PBR-^B'^P]x  -  0  (9.19) 

which  is  clearly  seen  to  be  the  sum  of  two  positive  semidefinite  terms,  recalling  that  Q  —  H^H,  Thus, 
for  (9.19)  to  be  zero,  we  must  have  both  terms  equal  to  zero.  By  also  noting  that  u  —  -R"^B^ Px^ 
(9.19)  can  be  rearranged  to  give 

[HxfiHx)  +  u^Ru  =  0  (9.20) 
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and  recalling  that  i?  >  0,  we  thus  must  have  x  e  Af[H]  and  u  =  0  for  (9.20)  to  hold,  so  that  what  we 
really  desire  to  identify  are  the  invariant  sets  of  the  continuous  time  open  loop  dynamics  contained 
in  the  nullspace  of  if,  i.e.,  all  x  such  that  x  e  Af[H]  and  x  is  invariant  with  respect  to 

X  =  a{x)  =  A{x)x  (9.21) 

Such  sets  may  occur  in  two  ways:  as  the  result  of  subsets  of  observable  states,  or  as  the  result  of 
subsets  of  unobservable  states.  By  unobservable  states  we  mean  invariant  manifolds  in  the  state 
space  on  which  y  does  not  change,  even  though  the  states  x  are  changing.  Such  invariant  manifolds 
admit  a  coordinate  transformation  so  that  the  output  does  not  depend  on  the  unobservable  states, 
where  the  number  of  unobservable  states  is  equal  to  the  dimension  of  the  manifold  [26].  Thus, 
nontrivial  invariant  unobservable  manifolds  will  have  dimension  one  or  greater,  so  that  manifolds 
in  the  observable  space  on  which  the  output  remains  constant  will  consist  of  only  single  isolated 
points  (zero  dimensional  manifolds).  We  talk  about  subsets  of  the  above  manifolds  because  even 
if  the  output  remains  constant  on  such  a  manifold,  we  still  are  interested  in  sets  where  h  remains 
constant  and  equal  to  zero.  The  result  of  this  is  that,  for  a  completely  observable  system,  even  if 
we  regulate  the  output  to  zero,  we  still  must  ensure  that  isolated  invariant  points  such  that  h  =  0 
do  not  exist,  or  else  we  cannot  guarantee  that  all  the  states  are  driven  to  zero.  For  a  system  with 
a  nontrivial  unobservable  space,  even  if  all  observable  states  are  driven  to  zero  by  regulation  of  the 
output,  the  remaining  unobservable  states  do  not  affect  the  output  and  so  are  not  compelled  to  go  to 
zero,  and  thus  closed  loop  stability  may  be  lost.  We  must  therefore  identify  the  unobservable  states 
and  ensure  all  trajectories  constrained  to  the  unobservable  manifold  converge  to  zero.  It  is  known 
how  to  characterize  unobservable  manifolds,  and,  in  fact,  in  [33],  a  recursive  algorithm  for  this  is 
given.  We  repeat  it  here,  referring  the  reader  to  [33]  for  additional  details.  Let  Ti{x)  be  covector 
fields  spanning  the  row  space  of  dh{x)  —  ||  for  all  x,  and  let  Qq  =  span[Ti],  Then  let 

Op  =  Hp-i  -j-  Ladp^i  (9.22) 

where  La^p~i  represents  the  Lie  derivative  of  each  covector  field  in  Qp-i  with  respect  to  a,  and  the 
H-  sign  represents  the  subspace  sum,  i.e.,  the  sum  of  the  spans.  If  the  above  procedure  converges  so 
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that  flp  =  fip-i  for  some  p,  then  the  unohservable  states  of  (9.21)  through  h  are  a  subset  of 

Oti  =  {a;  €  7^”|  ilpX  =  0}  (9.23) 

which  is  the  nullspace  of  a  matrix  function  of  a;  (a  distribution),  so  that  these  sets  may  vary  from 
point  to  point  in  the  state  space.  However,  if  the  dimension  d  of  Up  is  constant  for  all  x,  i.e..  Up  is 
a  nonsingular  distribution,  then  0:^i  will  be  a  smooth  manifold  of  dimension  n  —  d,  and  the  above 
mentioned  transformation  to  invariant  observable  and  unobservable  states  may  be  performed.  It 
should  be  stressed  that  nonsingularity  of  Up  is  critical  to  construction  of  the  invariant,  observable 
and  unobservable  manifolds.  If  Up  is  singular  at  some  points  in  the  state  space,  the  decomposition 
might  still  be  performable,  but  now  requires  additional  considerations  such  as  using  smt{Up),  the 
largest  smooth  distribution  contained  in  flp,  in  place  of  flp  itself  [33].  In  either  case  it  is  known  that 
fln_i  gives  a  characterization  of  the  observable  space  Oni  on  an  open  and  dense  subset  of  7?^”,  so 
that  often  p  =  n  in  (9.23).  By  assuming  this  and  applying  some  properties  of  Lie  derivatives,  we 
can  express  (9.23)  alternatively  as 

dHx 

dLlHx 

0^1  (9.24) 

dLl-^Hx 

where  is  a  shorthand  notation  for  differentiation  i  times  along  the  vector  field  a.  We  thus  see 
that  a  sufRcient  condition  for  there  not  to  exist  any  invariant  sets  of  (9.21)  unobservable  through  h 
other  than  x  =  0  is  for  the  rank  of  (9.24)  to  be  equal  to  n,  the  dimension  of  the  state  space,  for  all 
X.  This  leads  us  to  our  first  sufficient  condition  for  asymptotic  closed  loop  stability  with  Q  >  0  as 
opposed  to  (5  >  0,  as  stated  below. 

Theorem  9.3.2  Consider  (8.1)  written  as  (8.2),  with  the  system  matrices  in  (8.2)  globally  stabi- 
lizable/ detectable  and  analytic  with  respect  to  x.  Assume  the  sampled  data  SURE  control  law  (8.6), 
(8.5)  is  applied  to  (8.2)  where  Sk  is  selected  appropriately  small  to  guarantee  dominance  of  first-order 
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terms  in  the  expression  for  A14?  Q>ssume  linifc^oo  0?  further  assume  rank[Qp]  =  n  for  some 
p>  Q  and  for  all  x.  Then  the  closed  loop  system  is  asymptotically  stable. 

Proof:  The  proof  follows  directly  from  Theorem  9.3.1,  noting  that  the  assumptions  guarantee  that 
the  set  M  consists  only  of  the  zero  vector.  The  assumption  on  Sk  takes  the  given  form  because  Sk  is 
guaranteed  positive  for  any  finite  ifc,  and  thus  it  cannot  cause  AI4  to  equal  zero  except  as  k  tends  to 
infinity.  Now,  recall  that  it  was  mentioned  that  invariant  sets  in  the  nullspace  of  H  might  also  occur 
on  subsets  of  the  observable  space.  However,  by  definition  such  sets  must  consist  of  isolated  points, 
since  otherwise  they  would  belong  to  locally  invariant  unobservable  manifolds,  and  thus  would  not 
be  in  the  observable  space.  For  isolated  points  to  be  invariant  and  in  the  nullspace  of  H  we  must 
have  at  such  points  that  h  =  Hx  —  0  and  h  =  [H  +  ^]Ax  =  0,  as  well  as  having  higher-order 
derivatives  equal  to  zero.  However,  it  may  be  shown  that  for  analytic  systems  such  cannot  be  the 
case  if  rank[flp]  —  n.  This  is  a  consequence  of  an  analytic  function  being  zero  at  a:  —  0  and  having 
nonzero,  continuous  gradient  everywhere  else.  Thus,  Theorem  9.3.2  precludes  the  need  to  consider 
isolated  invariant  points  in  the  nullspace  of  so  that  the  theorem  does  indeed  provide  a  sufficient 
condition  for  asymptotic  closed  loop  stability.  ■ 

At  this  point  we  observe  that  there  are  two  sources  of  possible  conservatism  in  the  above  theorem. 
Note  first  of  all  that  if  h  is  a  linear  function  of  then  the  first  entry  in  is  iJ,  so  that  any  x  in 
Qp  necessarily  also  has  h  =  Hx  ~  0,  and  all  the  unobservable  states  are  also  in  the  nullspace  of  H. 
However,  H  h  is  not  a  purely  linear  function  of  a;,  then  the  first  entry  of  (Ip  is  dHx  =  H  ^  so 
that  r  E  does  not  necessarily  imply  that  x  is  in  the  nullspace  of  H.  The  result  of  this  is  that  for 
nonlinear  h  the  invariant  unobservable  sets  in  the  nullspace  of  H  truly  are,  in  general,  a  subset  of 
the  overall  invariant  unobservable  manifold,  and  thus  the  unobservable  manifold  may  contain  points 
not  in  the  set  S,  which  is  what  we  are  trying  to  identify.  However,  restricting  a  set  to  contain  only 
the  zero  vector  as  Theorem  9.3.2  does  also  necessarily  restricts  any  of  its  subsets  to  equal  the  zero 
vector.  The  second  source  of  possible  conservatism  is  that  the  above  analysis  assumes  that  u  =  0, 


9«9 


and  proceeds  to  identify  invariant  sets  based  on  that  assumption.  It  is  therefore  of  interest  to  try 
to  identify  when  this  assumption  is  satisfied;  i,e.,  for  what  x  is  u{x)  —  0? 


9.3,4  Factored  Observability  and  Control  Action 

The  analysis  is  aided  by  decomposition  of  (8.2)  into  linearly  as  opposed  to  nonlinearly  observable 
and  unobservable  parts,  where  linear  here  means  in  terms  of  the  pointwise  linearization  matrices 
A{x)  and  H{x).  It  is  well  known  from  linear  systems  theory  [41],  that  the  LTI  system 

X  =  Ax  -\-  Bu 

z  =  Hx  (9.25) 


has  an  invariant,  unobservable  space  given  by 


H 

HA 


HA^ 


-1 


(9.26) 


and  that  for  {A,i5}  stabilizable,  x  e  iff  rc  €  Af[P]  [50],  where  P  is  the  positive  semidefinite 
stabilizing  solution  to  the  algebraic  Riccati  equation 


A^P  A  PA-  PBR-^B'^P  +  Q  =  0 


(9.27) 


Thus,  Px  —  0  for  every  x  G  so  that  u{x)  —  —R^^B^Px  “  0  for  all  x  6  It  is  also  true 
[33]  that  for  an  LTI  system,  (9.24)  simplifies  to  (9.26),  so  that  is  an  invariant  set  of  (9.25)  in 
which  u  ~  0  and  Hx  —  0.  Thus,  for  an  LTI  system,  as  defined  in  (9.26)  completely  defines 
the  unobservable  subspace,  which  by  Lasalle’s  Invariance  principle,  requires  additional  assumptions 
for  stability.  In  LTI  linear  quadratic  regulator  theory  this  is  handled  by  requiring  the  invariant 
unobservable  dynamics  to  be  stable  (the  detectability  assumption).  If  we  now  generalize  (9.26)  to 
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state-dependent  factorizations,  we  find  that  if  x  belongs  to  the  set 


H{x) 


Of  =u 


H{x)A{x) 


(9.28) 


then  we  still  have  u{x)  —  0  and  H{x)  —  0,  but  we  are  not  guaranteed  that  Of  is  an  invariant  set  of 
(9.21)  because  of  possible  discrepancies  between  Of  and  Ofi,  These  discrepancies  arise  as  a  result 
of  a  general  nonequivalence  between  the  H{x)A^{x)  and  of  the  Lie  derivatives  dU^Hx^  and  hence  of 
their  nullspaces  (see  Chapter  6  for  proofs  of  nonequivalence  for  the  dual  case  of  controllable  sets). 
Note  that  the  above  analysis  again  points  to  the  necessity  of  precluding  isolated  invariant  points  in 
the  nullspace  of  iJ,  for  if  x  is  such  that  A{x)x  =  0  and  H{x)x  =  0,  i.e,,  x  is  an  open  loop  equilibrium 
in  the  nullspace  of  i/,  then  clearly,  x  G  Of  and  x  e  M.  As  an  immediate  consequence  of  these 
arguments  we  obtain  the  following  theorem  regarding  a  necessary  condition  for  global  asymptotic 
stability  of  SDRE  nonlinear  regulators  with  positive  semidefinite  state  weighting  matrices. 


Theorem  9.3.3  Consider  (8.1)  written  as  (8.2).  Suppose  multiple  open  loop  equilibria  exist  such 
that  a{x)  =  A{x)x  =  0  for  x  ^  0,  and  define  Aq  to  be  the  set  of  all  such  x  ^  0.  Then,  in  order  for 
the  closed  loop  system  (9.1)  to  be  globally  asymptotically  stable,  we  must  have  Hx  7^0  Vo;  6  Aq. 

Proof:  The  proof  follows  from  observing  that  x  E  Aq  and  Hx  =  0  implies  x  6  Of,  so  that 
u{x)  =  0.  Thus,  we  have  F{x)x  =  A{x)x  +  B{x)u{x)  =  0  so  that  x  E  M  and  in  fact  a:  is  a  closed 
loop  equilibrium  point  other  than  aj  =  0,  contradicting  global  asymptotic  stability  of  the  origin.  ■ 

We  observe  that  choosing  a  globally  nonsingular  H{x)  obviously  satisfies  the  requirements  of  Theo¬ 
rem  9.3.3,  preventing  any  difiiculties  from  multiple  open  loop  equilibria.  For  singular  H{x),  we  have 
the  following  sufficient  condition  for  satisfaction  of  the  requirements  of  Theorem  9.3.3. 

Theorem  9.3.4  Consider  (8.1)  written  as  (8.2),  and  assume  {H{x),A{x)}  is  detectable  for  all  x. 
Then  the  necessary  condition  for  global  asymptotic  stability  given  in  Theorem  9.3.3  is  satisfied. 
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Proof:  Recall  that  global  detectability  of  {H{x),A{x)}  implies  that  for  all  X,x  and  such  that 

A{x)z  =  Xz  and  Re{X)  >  0,  H{x)z  7^  0.  Let  A  -  0.  Then  A(x)z  —  0  implies  H{x)z  ^  0,  so  that  the 
theorem  is  proven.  ■ 


P.  5, 5  Characterization  of  M 

To  this  point  we  have  seen  that  if  rr  E  Of,  then  u{x)  —  0.  However,  we  do  not  necessarily  have  the 
reverse  implication,  which  means  that  we  do  not  know  the  set  of  all  x  such  that  u{x)  =  0,  and  thus 
we  do  not  know  which  x  €  Ofi  are  actually  troublesome  from  a  stability  viewpoint.  However,  if  we 
assume  that  the  system  is  nonlinearly  stabilizable  and  Sk  remains  nonzero,  we  do  not  have  to  be 
concerned  with  x  outside  of  Of,  as  proven  in  the  following  theorem. 

Theorem  9,3.5  Consider  (8.1)  written  as  (8.2),  assume  liuik^ooSk  /  0,  and  assume  that  (8.1)  is 
nonlinearly  stabilizable,  as  defined  in  Section  J^.l.  Then  the  set  M  in  Theorem  9.3.1  for  the  SDRE 
regulator,  i.e.,  the  union  of  invariant  sets  of  (9.21)  contained  in  the  nullspace  of  H,  is  contained  in 
the  intersection  of  Of  and  Ofi,  which  we  call 

Proof:  We  prove  only  the  case  of  singular  H{x)  for  some  x  ^  0,  since  the  case  for  globally 

nonsingular  ff(x)  is  trivial,  the  intersection  being  the  zero  vector.  Obviously,  if  x  is  an  element  of 
both  Of  and  Ofi,  then  Hx  =  0  and  u{x)  =  0  so  that  x  e  E  and,  in  fact,  x  remains  in  the  nullspace 
of  H  under  (9.21),  so  that  x  E  M.  Now,  we  have  previously  shown  that  M  C  Ofi.  Prom  this  it  is 
clear  that  we  cannot  have  x  E  Of  and  x  E  M,  without  having  x  E  Ofi  as  well.  We  therefore  need 
to  prove  only  that  x  cannot  be  an  element  of  both  M  and  Ofi  without  being  an  element  of  Of , 
assuming  (8.1)  is  nonlinearly  stabilizable.  Thus,  assume  x  E  Ofi  and  x  E  M.  This  implies  Hx  —  0, 
u{x)  =  0  and  x  remains  in  the  nullspace  of  H  under  (9.21).  Now,  suppose  x  is  not  in  Of.  Note  that 
this  excludes  any  x  E  Aq,  where  ^0  is  defined  as  in  Theorem  9.3.3,  since  any  E  ^0  is  automatically 
in  both  Ofi  and  Of  if  Hx  —  0  as  well.  Thus,  A{x)x  /  0  so  that  x  may  not  be  an  isolated  closed 
loop  equilibrium  point,  and  also  we  have  P{x)x  /  0,  but  u{x)  —  0.  Now,  recalling  u  —  —R~^B^Px, 
we  see  that  rt  =  0  iff  B^Px  =  0,  since  R  is  globally  nonsingular.  From  the  previous  discussion  we 
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have  X  ^  0  and  Px  ^  0,  so  that  Px  ^  0  must  belong  to  an  invariant  set  of  (9.21)  in  the  nullspace  of 

.  Since,  alternatively,  this  is  an  invariant  set  of  (9.21)  such  that  x^PB  ■=  0,  we  see  that  this  may 
actually  be  considered  a  nonlinear  controllability  question.  If  (8.1)  is  weakly  controllable  on  T?.””, 
then  there  is  no  nonzero  vector  —  x^P  which  remains  in  the  left  nullspace  of  B  for  all  x  under 
(9.21).  Thus,  global  weak  controllability  of  (8.1)  may  be  assumed  in  place  of  the  weaker  assumption 
of  nonlinear  stabilizability  in  the  theorem  statement.  Nonlinear  stabilizability  of  (8.1),  however,  is 
sufficient  to  guarantee  that  trajectories  contained  in  the  left  nullspace  of  B  converge  to  zero.  Hence 
x'^P  converges  to  zero,  which  is  a  contradiction,  so  that  no  such  x  outside  of  Oj-  may  be  in  M,  and 
the  theorem  is  proven.  ■ 

This  leads  us  to  additional  sufficient  conditions  for  asymptotic  closed  loop  stability  with  Q  >  0, 
as  given  in  the  following  theorems. 

Theorem  9,3.6  Consider  (81)  writteri  as  (8,2) ^  with  (8.1)  nonlinearly  stabilizable  and  with  the 
system  matrices  in  (8.2)  globally  stabilizable/ detectable  and  analytic  with  respect  to  x.  Assume  the 
sampled  data  SDRE  control  law  (8.6)^  (8.5)  is  applied  to  (8.2)  where  6k  is  selected  appropriately  small 
to  guarantee  dominance  of  first-order  terms  in  the  expression  for  AVk,  assume  limfc_>oo  Sk  /  0,  and 
further  assume  Oj-  =  0.  Then  the  closed  loop  system  is  asymptotically  stable. 

Proof:  The  proof  follows  from  Theorem  9.3.1  analagously  to  that  of  Theorem  9.3.2,  where  we  have 
ensured  M  =  0  by  the  assumptions  of  nonlinear  stabilizability  and  Oj  =  0  and  invoking  Theorem 
9.3.5.  ■ 

Theorem  9.3.7  Consider  (8.1)  written  as  (8.2),  with  (8.1)  nonlinearly  stabilizable  and  with  the 
system  matrices  in  (8.2)  globally  stabilizable/ detectable  and  analytic  with  respect  to  x.  Assume  the 
sampled  data  SDRE  control  law  (8.6),  (8.5)  is  applied  to  (8.2)  where  6k  is  selected  appropriately  small 
to  guarantee  dominance  of  first-order  terms  in  the  expression  for  AVk-  Assume  limfc^oo  Sk  /  0,  and 
further  assume  =  0.  Then  the  closed  loop  system,  is  asymptotically  stable. 
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Proof:  The  proof  follows  from  Theorem  9.3.1  analagoiisly  to  that  of  Theorem  9.3.6,  where  we  have 
ensured  Af  —  0  by  the  assumptions  of  nonlinear  stabilizability  and  Iqi.  =  0  and  invoking  Theorem 

9.3.5.  ■ 


Theorem  9.3^8  Consider  (8J)  written  as  (8,2),  with  (8,1)  nonlinearly  stabilizable  and  with  the 
system  matrices  in  (8,2)  globally  stabilizable/ detectable  and  analytic  with  respect  to  x.  Assume  the 
sampled  data  SURE  control  law  (8.6),  (8,5)  is  applied  to  (8.2)  where  6k  is  selected  appropriately  small 
to  guarantee  dominance  of  first-order  terms  in  the  expression  for  AVk,  assume  lim^-^oo  /  0,  and 
further  assume  that  for  x  restricted  to  (9.21)  is  asymptotically  stable.  Then  the  closed  loop 
system  is  asymptotically  stable. 

Proof:  The  proof  follows  from  Theorem  9.3.1  analagously  to  that  of  Theorem  9.3.7,  where  the 

assumptions  ensure  that  trajectories  confined  to  M  are  asymptotically  stable.  ■ 

9.3.6  The  Effect  of  Factored  Controllability 

The  above  analysis  shows  that,  as  long  as  the  system  considered  is  nonlinearly  stabilizable,  poten¬ 
tially  troublesome  sets  from  a  stability  perspective  arise  from  the  invariant  open  loop  unobservable 
(unpenalized)  states.  Nonlinear  stabilizability  also  allows  this  set  of  concern  to  be  reduced  by  exclud¬ 
ing  x's  not  also  in  the  factored  unobservable  space.  From  these  statements  we  see  that  nonlinear 
controllability  (alternatively,  stabilizability)  plays  an  important  role  in  our  stability  analysis.  To 
conclude  this  discussion,  let  us  make  a  comment  regarding  factored  uncontrollable  sets  of  (8.2), 
and  their  impact  on  closed  loop  stability  analysis.  Analagous  to  the  above  derivation  of  observable 
and  unobservable  sets,  it  is  possible  to  obtain  characterizations  of  factored  controllable  and  uncon¬ 
trollable  sets  for  (8.2)  (see  Chapter  6).  In  fact,  it  can  be  shown  [41]  that  based  on  the  factored 
uncontrollable  set  defined  by 

cf  =  {xe  n^\  x'^Mcfix)  -  x'^{B{x)  A{x)B{x)  ■  ■  ■  Aix^-^^ Bix)]  =  0}  (9.29) 
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(8.2)  may  be  decomposed  at  each  x  into  the  form 


=  Aii{x)xi  4-  Ai2{x)x2  +  Bi{x)u 

^2  "  A22{^2)^2  (9.30) 

where  X\  is  in  the  factored  controllable  subspace  Cf,  X2  is  in  the  factored  uncontrollable  subspace 
Cj-,  and  {Aii(x)^  Bi{x)}  is  a  controllable  pair.  Thus,  if  x  E  then  x^B{x)  =  0  so  that  x 
has  a  component  unaffected  by  the  control  {x2)j  and  this  component  could  be  troublesome  from  a 
stability  point  of  view  if  the  above  transformation  yields  an  invariant  manifold  within  the  pointwise 
X2  space,  and  the  appropriate  open  loop  dynamics  confined  to  this  manifold  are  not  stable.  This 
eventuality  is  precluded,  however,  by  the  assumption  of  nonlinear  stabilizability.  The  impact  of  a 
globally  stabilizahle  parametrization  on  these  issues  is  that,  analagous  to  Theorem  9.3.4,  using  such 
a  factorization  prevents  isolated  uncontrollable  points  from  being  invariant.  This  is  because  such 
a  globally  stabilizahle  parametrization  guarantees  that  any  x  such  that  x'^B  —  0  is  such  that  Ax 
is  nonzero,  so  that  such  an  x  does  not  belong  to  the  set  and  hence  not  to  M  either,  unless  it 
belongs  to  a  larger,  invariant,  uncontrollable  set.  Note  that  uncontrollable  points  as  determined 
from  (9.29),  (9.30)  which  are  also  in  add  nothing  new  to  the  analysis,  since  the  effect  of  0:^i  has 
already  been  taken  into  account.  From  the  above  considerations,  we  see  that  uncontrollable  states 
are  important  from  a  stability  analysis  viewpoint,  but,  given  a  globally  stabilizahle  factorization, 
they  are  determined  from  the  true  nonlinear  controllability,  and  not  from  the  factored  system.  Their 
impact  is  in  determining  whether  Pk  converges  for  x  E  Of ,  identifying  what  states  must  be  placed  in 
the  nullspace  of  H  due  to  lack  of  controllability /stabilizability,  and  whether  any  such  uncontrollable 
dynamics  in  M  are  stable.  To  summarize,  we  have,  assuming  {A(a:),  ^(a;)}  stabilizahle  for  all  x, 
that 

•  if  a’  €  Cf  and  x  E  then  x  may  be  in  M  as  determined  by  Theorem  9.3.5.  Overall  stability 
requires  stability  of  open  loop  trajectories  contained  in  M  (zero  state  detectability); 

•  U  x  E  Cf,  and  x  is  not  in  then  x  is  not  in  M; 


9-15 


•  if  the  system  is  nonlinearly  stabilizable,  then  O^i  in  the  above  may  be  replaced  with  Of  or 
Iql.  In  this  case,  if  x  is  in  the  true  invariant  nonlinearly  uncontrollable  subspace,  then  similar 
to  the  second  item  above,  r  must  be  in  Of  for  global  closed  loop  stability  to  hold. 

9.3.7  Convergence  of  Pk  and  Sk  ^0 

Finally,  let  us  consider  the  condition  s{x)  =  0  of  Section  9.3.2.  By  definition,  is  either  equal 
to  the  value  of  s  at  the  previous  sampling  time,  or  the  ratio  of  two  positive  numbers  times  5^. 
Thus,  as  long  as  tk  is  finite,  there  are  a  finite  number  of  positive  terms  in  the  recursion  for  5^,  so 
that  s{xr)  is  finite,  bounded,  and  positive.  The  same  is  true  if  Xk  converges  to  zero.  By  (9.11),  if  x 
becomes  small  enough  (enters  the  domain  of  attraction  of  the  linearized  closed  loop  system,  which  is 
guaranteed  asymptotically  stable  near  the  origin),  then  P  becomes  and  remains  essentially  constant 
so  that  (9.10)  will  no  longer  hold.  At  some  point  in  this  case  Sr  =  Sk  for  all  U  from  that  time  on, 
where  Sk  has  been  computed  as  a  product  of  a  finite  number  of  positive  terms.  We  therefore  are 
concerned  only  with  preventing  s  {xk )  from  tending  to  zero  as  4  oo  for  positive  definite  Pk  and 
Xk  not  tending  to  zero,  as  stated  in  the  above  theorems.  We  restrict  ourselves  to  positive  definite 
Pk  for  sufficiently  large  k  because  if  Pk  is  only  positive  semidefinite  and  we  have  convergence  to  an 
X  ^  0  in  the  nullspace  of  Pk ,  then  we  already  know  that  the  Lyapunov  variation  A V*  will  equal 
zero,  and  we  do  not  care  if  Sk  tends  to  zero  or  not.  Thus,  we  now  show  that  Sk  will  not  tend  to  zero 
under  the  assumptions  of  global  analyticity  and  stabilizability /detectability  of  the  system  matrices 
and  convergence  of  to  F  >  0,  with  assumed  constant  sampling  interval. 

Theorem  9,3.9  Consider  (8.1)  written  as  (8.2),  with  the  system  matrices  in  (8.2)  globally  stabi¬ 
lizable/ detectable  and  analytic  with  respect  to  x.  Assume  that  the  sampled  data  SDRE  control  law 
(8.6),  (8.5)  is  applied  to  (8.2)  where  6k  =  8  is  selected  appropriately  small  to  guarantee  dominance 
of  first-order  terms  in  the  expression  for  AVk,  and  assume  that  the  closed  loop  trajectory  converges 
to  a  point  in  Of,  so  that  lim^_^ooPfc  =  P  >  0  exists.  Then  limA:->oo  exists  and  is  a  positive  number 
s  >  0. 
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Proof:  As  discussed  above,  the  proof  is  trivial  if  the  trajectory  converges  in  a  finite  number  of 

sampling  intervals  or  if  it  converges  to  the  origin.  In  fact,  we  only  need  consider  when  (9.10)  holds 
for  tk  tending  to  infinity,  or  equivalently  when  xJ^P^Xk  >  xJlQk  PkBkRk^ Pk]xk  for  fc  — >  oo. 


Using  (9.10)  we  may  thus  write 

xl_iPk-lXk-l 

X^_lPkXk  —  l 


Sk 


-Sfc-1 


x'k_^Pk  —  lXk  —  l  X^_2Pk—2Xk—2  x'qPqXq  ^ 

xl^^PkXk-l  xl_2Pk-l^k-2  xIPiXq 

^  T  T> 

_  jjj-  Xj^_^Pi—iXi—i 


(9.31) 


ii  -T 

i=l  *^1-1 

so  that  as  k  tends  to  infinity,  we  are  trying  to  guarantee  convergence  of  an  infinite  product  of  nonzero 
(positive)  terms.  If  we  now  consider  the  last  line  of  (9.31),  for  suSiciently  small  6  we  may  use  the 
analyticity  of  F  to  write  Pi  ^  Pi^i  -h  ^F/_i  or  equivalently  Pi-i  ^  Pi-  SPl_i.  Substituting  this  into 
(9.31)  and  dividing  both  the  numerator  and  denominator  term  by  term  by  xf_^PiXi-ij  and  letting 
i  tend  to  infinity,  (9.31)  becomes 

oo 

lira  s*  =  0(1  +  aj)  (9.32) 

fc^oo 


where 


cii  —  S' 


i=l 


xJ_iFiXi-i 


(9.33) 


Now,  a  well  known  [1]  sufficient  condition  for  convergence  of  (9.32)  to  a  nonzero  value  is  absolute 
convergence  of  the  infinite  series 

oo 

(9.34) 

or,  equivalently,  convergence  of 

oo 

(9.35) 

i=l 

This  series  is  in  turn  known  to  converge  if  it  passes  the  ratio  test  [1] 

«^i+l 


R  =  lim  sup 

i— >oo 

Using  (9.33)  in  (9.36)  we  thus  seek  to  prove 

R  —  lim  sup 


CLi 


<  1 


(9.36) 


T 

X;  i 

i^i 

x'f  if 

^iXi^l 

<  1 


(9.37) 
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or 


xJ_iPiXi-i 

Now  recall  that  we  only  need  to  show  (9.38)  holds  when 


lim  sup 

i—>oo 


X  PJo 


xJPi+iXi 


<  lim  inf 


xJP'iXi  >  xJlQi  +  PiBiRi  ^BfPi\x, 


(9.38) 


(9.39) 


holds,  and  we  know  xi  ^  x  0  and  ^  P  >  0,  so  that  for  sufficiently  large  the  right  hand 
side  of  (9.39)  is  positive  and  thus  the  numerators  and  denominators  of  both  sides  of  (9.38)  are  also 
positive,  allowing  the  absolute  value  signs  in  (9.38)  to  be  removed.  Thus  xJPlxi  >  0  is  a  positive 
sequence  which  we  know  must  converge  to  zero  since  the  Pi  converge,  so  that  the  ai  satisfy  the 
necessary  condition  for  convergence  of  an  infinite  series,  limi_^oo  =  0-  Now,  since  P'  and  x  are 
analytic  we  see  that  for  sufficiently  large  i  the  values  of  xfPIxi  >  0  fall  along  a  decreasing  line  going 
through  zero.  Thus,  we  have 

xJPlxi  =  xf_iPl_iXi-i  -  Ci-i  (9.40) 

where  is  a  small  positive  number.  If  we  again  make  use  of  the  analyticity  of  P  and  x  to  write 


xJPiXi  -  xJ^^Pi-iXi-i  8ixJ_^[Pl_-^  +Pi„iFi_i  +  +  xJ_iO{8‘l)xi-i 

~  xJ_iPi-iXi^i  +  SiX^_-^\P —  Qi-l  “  Pi-lPi-lPi-iPi— lPi-l]^i-l 

-\-xJ_^0{8’f)xi-i  (9.41) 


then  positivity  of  (9.39)  and  smallness  of  8i  allow  us  to  write 

xJPiXi  =  xJ„iPi-.iXi-i  -f-r’i-i 


(9.42) 


where  Vi-i  is  again  a  small  positive  number.  Thus,  using  (9.40)  and  (9.42)  we  see  that  (9.38)  reduces 
to 


X^_^Pi—\Xi~\  ”{“  Vi—i  Xj^_^Pi—xXi—}^ 

which  clearly  holds  so  that  the  Ratio  test  is  satisfied,  and  the  theorem  is  proven. 


(9.43) 
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We  conclude  this  section  by  stating  that  the  assumption  of  lim^-^oo  0  found  in  all  our  earlier 
theorems  may  thus  be  replaced  by  the  assumption  that  Pk  converges  to  a  positive  definite  matrix 
limit. 


Examples 

We  now  illustrate  the  preceding  theorems  by  means  of  some  simple  examples. 
Example  1 

Consider  the  system 


xi  -  X1X2  -i-ui 
X2  =  -h  U2 
h  ~  X1X2  ~\-X2 


(9.44) 


with  R  —  I.  Note  that  the  set  of  open  loop  equilibrium  points  Aq  equals  the  X2  axis,  i.e,  all  x  such 
that  xi  =  0.  Also  note  that  the  set  where  ?/  =  0  equals  the  union  of  the  xi  axis  and  the  line  Xi  =  —1. 
Thus,  for  any  H  we  choose,  there  can  be  no  a;  0  both  in  Aq  and  the  nullspace  of  if,  so  that  the 
conditions  of  Theorem  9.3.3  are  satisfied.  If  we  choose 


A[x)  ^ 


x-i  0 

1  0 


(9.45) 


and  H{x)  =  [x^  1],  then  we  find 

Of  = 


H{x) 

X2  1 

H{x)A{x) 

lAxl  0 

(9.46) 


SO  that  Of  is  full  rank  for  all  x^  and  thus  Of  ”  {0}.  Let  us  now  construct  Qp  according  to  the 


recursive  procedure  given  in  Section  9.3.  We  have 

fto  =  rowspan  dh  =  rowspan  [x2  xi  -h  1] 
0,1  =  Oq  dL^h 


(9.47) 

(9.48) 
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A  simple  computation  gives 


=  rowspan 


X2 


(9.49) 


1^  x\^2xi^-l  2xiX2  J 

which  equals  the  observable  space  on  at  least  an  open  and  dense  subset  of  TZ^ .  Clearly,  contains 
nontrivial  elements,  for  example,  {xi,X2)  ~  (-1,0).  However,  since  Of  =  {0}  we  have  satisfied 
Theorem  9.3.3,  and  Theorem  9.3.6  tells  us  is  irrelevant  and  that  bounded  trajectories  can  only 
converge  to  the  origin,  so  that  closed  loop  stability  is  guaranteed  (provided  we  sample  fast  enough 
and  the  SDRE  solution  converges).  Additionally  we  note  that  for  this  system  B  =  I  is  full  rank 
and  constant,  which  implies  the  system  is  controllable  for  all  x  (both  in  a  factored  and  nonlinear 
sense),  and  thus  the  algorithm  is  globally  well-defined.  This  example  thus  shows  that  the  factored 
observability  can  play  a  crucial  role  in  SDRE  application.  In  the  next  example  we  illustrate  the 
importance  of  Theorem  9.3,3. 

Example  2 

Consider  the  system 


Xi  =  X1X2  X2  -\-ui 
±2  =  X2-\~  U2 


h  =  X1X2  -f-  X2 


(9.50) 


again  with  R~  I.  The  system  has  the  same  control  input  matrix  B  —  I  and  output  h  as  in  Example 
1,  but  we  have  altered  the  open  loop  dynamics  a.  Now  we  see  that  Ax  =  0  for  all  x  such  that  X2  —  0, 
and  h  =  0  also  at  such  points.  This  means  that  for  any  choice  of  A  and  H  we  cannot  satisfy  Theorem 
9.3.3,  and  the  closed  loop  system  will  contain  equilibrium  points  all  along  the  xi  axis.  Thus,  even 
though  we  have  full  controllability  of  this  system,  the  dynamics  and  chosen  output  combine  in  such 
a  way  that  any  trajectory  either  starting  on  or  passing  through  the  Xi  axis  will  be  fixed  at  the  point 
of  intersection.  To  illustrate  this  discussion,  choose 


X2  1 
0  X2 


(9.51) 
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aad  h  as  in  Example  1.  Simple  computations  give 


0} 


X2  1 
0  X2 


(9.52) 


and 


fti 


X2  + 1 


(9.53) 


^  2x1  +  1)  J 

where  from  here  on  it  is  implicitly  understood  that  the  rowspans  of  the  above  distributions  are 
implied  by  the  above  notation.  As  predicted,  we  see  Of  ~  {x  \x2  =  0},  and  we  also  see  that 
—  {x  ^  V?  \  X2  =  ^  or  xi  =  —1],  The  intersection  of  the  two  is  thus  the  Xi  axis  as  discussed 
above.  In  these  first  two  examples  we  have  illustrated  the  importance  of  the  relationship  between  A 
and  iT  to  observability  and  hence  stability.  In  the  next  two  examples  we  also  incorporate  the  effects 
of  the  B  matrix,  so  as  to  illustrate  the  effects  of  controllability /stabilizability  as  well. 

Example  3 

Consider  the  system 


Xi  ^  XiX2-^U 


X2  ~  ”(1  H-  x\)x2 


h  —  xi 


(9.54) 


so  that  H  =  [10],  and  where  R=  1.  Note  that  the  intersection  between  the  open  loop  equilibrium 
points  (the  Xi  axis)  and  the  points  in  the  nullspace  of  H  (the  X2  axis)  consists  of  only  the  zero 
vector,  so  that  Theorem  9.3.3  holds.  Also  note  that  B  =  [1  0]^  and  that  for  =  0  we  thus  clearly 
do  not  have  full  controllability  of  X2 .  Choosing 


X2  0 

0  -(1  +  xj) 

we  see  that  {A^B}  is  stabilizable,  but  not  controllable  for  all  x.  For  the  given  H  we  find 


(9.55) 


Of  = 


1  0 
X2  0 


(9.56) 
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and 


1  0 


ill  = 


X2  2;i 


(9.57) 


so  that  the  intersection  of  Of  and  Ofi  can  be  no  bigger  than  the  X2  axis.  However,  along  the  X2 
axis  we  see  that  the  dynamics  reduce  to 


X2  —  -X2 


(9.58) 


so  that  by  Theorem  9.3.8  we  can  conclude  closed  loop  stability  for  sufficiently  rapid  sampling. 

Example  4 

In  this  example  we  illustrate  the  results  of  different  choices  of  H  for  a  given  system,  incorporating 
virtually  all  the  theory  developed  in  this  chapter.  Consider  the  system 


±1  —  X1X2  +  X2 


±2 —  u 

hi  =  [xi  X2]^  “  Hx  =  Ix 


h2  =  X2~  Hx  —  [0  l]x 


(9.59) 

(9.60) 


We  can  immediately  deduce  that  the  set  of  open  loop  equilibrium  points  ^0  =  ^  I  “ 

—I  or  X2  ~  0},  Thus,  taking  /i  =  /ii  we  satisfy  the  requirements  of  Theorem  9.3.3,  but  taking 
/i  —  /i2  we  have  that  the  Xx  axis  consists  entirely  of  closed  loop  equilibrium  points.  Now,  it  is  easily 
seen  that  the  above  system  has  an  invariant,  uncontrollable  set  which  is  Cf;i  —  {x  Eli?  \  Xi  =  —1}, 
so  that  any  trajectories  which  start  in  or  enter  remain  there  for  all  future  time.  However,  if  we 
choose 


A{x)  = 


X2  1 


(9.61) 


0  0 


then 


A{x)B 


1 

0 


(9.62) 
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which  gives  the  factored  controllability  matrix 


Mcf{x)  -  [B  AB]  = 


0  1 
1  0 


(9.63) 


This  matrix  is  full  rank  for  all  so  that  the  system  has  a  controllable  factorization  for  all  x.  Now, 
for  A  =  /ii  we  have  =  /  so  that  both  Of  —  [H^  A?-  and  Oni  —  [H^  where 

J  is  the  Jacobian  matrix  of  a,  are  clearly  full  rank  for  all  x.  Thus,  the  only  possible  closed  loop 
equlibrium  point  for  this  choice  of  regulated  output  is  the  origin.  However,  even  though  we  have  a 
globally  controllable  factorization,  the  lack  of  true  controllability  for  trajectories  passing  through  C^i 
prevents  them  from  reaching  the  origin.  Thus,  by  Lasalle’s  Invariance  Principle,  we  expect  any  such 
trajectory  to  be  unbounded.  Figure  9.1  shows  such  a  trajectory  which  starts  from  Xq  =  [—1  2]^, 
and  indeed,  we  see  the  X2  state  growing  unbounded  while  the  Xi  state  remains  fixed  at  —  1.  In 
Figure  9.2  we  give  time  histories  of  the  Lyapunov  function  V  and  the  scaling  factor  s.  Note  that, 
although  V  is  continuous  and  decreasing  for  all  time,  the  unbounded  growth  of  x  drives  s  to  zero  as 
time  increases. 

If,  however,  we  now  take  =  ^25  we  find 


Of  =  Onl  = 


0  1 
0  0 


(9.64) 


so  that  the  Xi  axis  is  unobservable  and  receives  no  control  action.  Thus,  bounded  trajectories 
will  converge  to  the  set  where  X2  =  0.  Technically,  such  a  choice  of  H  does  not  give  a  detectable 
factorization,  but  if  we  solve  the  SDRE  for  only  the  ±2  equation,  the  control  is  well-defined.  Doing  so, 
we  find  that  trajectories  that  intersect  C:^i  now  converge  to  the  only  allowable  closed  loop  equilibrium 
point,  X  =  [-1  0]^.  Figures  9.3  and  9.4  are  simulation  plots  illustrating  this  result.  Note  that  s  is 
well-behaved  in  this  simulation  due  to  the  convergence  of  Xk  and  and  in  fact,  x^Px  would  serve 
equally  well  as  a  Lyapunov  function  in  this  case  since  the  scaling  remains  identically  one.  For  both 
simulations  the  sample  rate  was  20  Hz. 
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Time  (s) 

Figure  9.2:  Histories  of  V  and  s  for  h  =  hi 
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9.5  Summary  and  Condnsions 


We  have  shown  that  it  is  possible  to  choose  positive  semidefinite  state  weighting  matrices  in  a 
nonlinear  SDRE  suboptimal  regulation  problem  and  still  maintain  closed  loop  stability,  provided 

•  the  system  is  globally  analytic  and  pointwise  stabilizable/detectable  with  respect  to  x 

•  the  system  is  sampled  and  controlled  often  enough 

•  the  SDRE  solution  converges 

Observability  of  both  the  tme  nonlinear  and  pointwise  linearized  systems  were  seen  to  play  a  key 
role  in  the  determination  of  invariant  sets,  while  controllability  issues  were  seen  to  affect  convergence 
of  the  SDRE  solution.  We  have  also  established  a  necessary  condition  for  global  asymptotic  stability 
of  the  closed  loop  system,  which  is  that  the  regulated  output  must  not  be  chosen  so  that  open  loop 
equilibrium  points  yield  a  zero  output  value.  We  then  showed  that  this  condition  is  automatically 
satisfied  if  ^(2^)}  is  detectable  for  all  x.  The  results  were  derived  from  a  discrete  time  version 

of  LaSalle’s  Invariance  principle  applied  to  the  sample  data  control  law  and  continuous  time  plant 
dynamics.  Examples  and  simulations  were  given  to  illustrate  and  verify  the  theory. 
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X.  Exponential  Stability  of  SDRE  Regulators 


10.1  Introduction 

In  this  chapter  we  investigate  potential  exponential  stability  of  SDRE  regulators.  Our  approach 
is  to  begin  by  looking  at  systems  whose  dynamics  can  be  described  by  piecewise  constant  linear 
representations  over  fixed  intervals,  and  then  extrapolate  the  results  to  SDRE  regulators  by  including 
perturbation  terms  in  the  dynamics.  Our  final  result  is  a  conjecture  suggesting  conditions  under 
which,  exponential  stability  of  SDRE  regulators  may  be  obtained. 

10.  S  Constant  Fk  Matrices 

We  wish  to  consider  stability  of  the  nonlinear  autonomous  dynamical  system 

x{t)  -  f{x)  -  F{x{t))x{t)  (10.1) 

with  state  vector  x  e  TZ'^,  and  with  F{x{t))  defined  to  be  equal  to  a  sequence  of  Hurwitz  matrices, 
Ffc^  which  are  assumed  constant  over  finite  time  intervals  [tk,  4+i)-  That  is, 

x{t)^Fkx{t)  V  t  e  [tk,tk-\.i)  (10.2) 

and  Re[Xi(Fk)]  <  0  V  i,  fc.  Note  that  for  all  practical  purposes  we  could  thus  consider  F{x{t))  to  be  a 
function  of  time  only,  so  that  (10.1)  would  be  linear  time-varying,  but  we  maintain  the  x  dependency 
for  later  use.  For  such  a  system  we  may  write  with  no  error 

Xk-\-l  —  (10.3) 

where  ^k  =  is  the  state  transition  matrix  from  4  to  4+i*  Let  us  define  6k  =  4-n  —tf. 

and  now  choose  any  initial  condition  x{to)  ~  Xq  eFF.  We  seek  to  discover  conditions  under  which 
the  norm  of  the  state  vector  will  eventually  converge  to  zero,  i.e.,  the  origin  is  attractive.  To  this 
end  we  consider  the  value  of  a^i,  which  from  (10.3)  may  be  written  as 

xi  ~  ^qXo  (10.4) 
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Defining  ||x||  to  be  the  vector  2-norm  (Euclidean  norm),  we  may  write 


|ki||<a(^o)|ko||  (10.5) 

where  a  represents  the  maximum  singular  value.  Now,  we  may  invoke  the  definition  of  $  and  the 
properties  of  the  exponential  of  a  Hurwitz  matrix  to  write  [42] 

a($o)  <  (10.6) 

for  constants  do>  Q  and  cq  >  1,  so  that  (10.5)  becomes 

Ikill  <coe-‘*»'o||a;o||  (10.7) 

and  we  see  that  the  norm  of  x  is  not  guaranteed  to  be  decreasing  over  the  first  time  interval  since 
Co  >  1.  If  we  now  consider  the  value  of  the  state  vector  at  t2  we  have 

a;2  =^1^02^0  (10.8) 

so  that  by  defining  c?i,  Ci  and  6i  for  as  in  (10.6)  we  have 

||2;2||  <coCie-^^*^e-''°^'>||a:o||  (10.9) 

If  we  now  let  A2  =  ^2  -  -H  ^0  it  is  clear  that  we  can  write 

\\x2\\<coCie-^^^^\\xo\\  (10.10) 

for  some  m2  >  0  (m2  =  |(di  -f  do)  if  =  So)-  By  induction,  after  r  time  steps  we  can  therefore 
write 

||2;,||  <CoCi...c,c---^^||a^o||  (10.11) 

Now  if  we  take  the  limit  of  (10.11)  as  r  ^  00,  the  exponential  term  tends  to  0  provided  m,. 
does  not  approach  0,  but  we  are  not  able  to  conclude  that  ^  0  because  the  infinite  product 
of  the  Cjfc’s  may  grow  faster  than  the  exponential  term  decays.  This  is  why  simply  guaranteeing 
negative  eigenvalues  in  a  closed  loop  matrix  for  a  linear  time- varying  system  is  insufiicient  to  conclude 
stability.  Negative  eigenvalues  of  the  Fk  guarantee  eigenvalues  of  inside  the  unit  disk,  but  not 
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that  ||#fc||  <  1.  Thus,  we  shall  return  to  (10.8)  and  seek  additional  conditions  on  the  Fk  beside  being 
Hurwitz  which  lead  to  attractiveness  of  the  origin.  By  using  the  Jordan  form  of  Fk  =  MkJkM^^, 
we  may  write  (10.8)  as 

X2  =  (10.12) 

Now,  if  we  define  AMi  =  Mo  -  Mi,  then  we  can  write 

X2  =  Mf\Mi  +  AMi)e-^°^°Mo^XQ 

=  Mie-’-^^^  e-’o^^M^'^xo  +  Mie-^'^'M{;^AMie-^^‘°Mo^xo  (10.13) 


At  this  point  we  assume  each  Fk  is  diagonalizable  (simple)  so  that  Jk  =  Dk,  &  diagonal  matrix  of 
the  eigenvalues  of  Fk,  all  of  which  have  negative  real  part  by  the  Hurwitz  assumption.  Equation 
(10.13)  thus  becomes 


X2  =  +  Mie^^^^  M^^AMie^^^^Mo^xo 

=  Mie^^^^Mo^Xo  +  Mie^^^^M{-^AMie^°^°MQ^xo  (10.14) 

where  D2  is  a  diagonal  matrix  of  possibly  complex  numbers,  all  of’which  have  negative  real  part.  In 
the  event  —  So,  we  have  B2  =  ^(Di  +  Bo),  so  that  B2  is  just  the  average  of  the  Bk-  If  we  now 
take  the  norm  of  (10.14),  use  the  triangle  inequality  and  the  submultiplicative  property,  and  invoke 
some  singular  value  properties  we  find 


IN2II  < 


(T^Mi) 

S.{Mo) 


ll^o||  + 


HMi)  a, s,  ^(AMllgdoSo 

g_{Mo)  3.{Mi) 


Ikoll 


(10.15) 


where 

m2o  =  max  |i?e[Ai(.D2)]|  >  0  (10.16) 

i 

and  di  and  do  are  defined  likewise  for  Di  and  i)o,  respectively.  Suppose  now  that  we  define  a  scalar 
valued  function  that  overbounds  the  error  term  due  to  the  change  in  Mk ,  so  that 

^<,(0  (10.17) 
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where  ri[k)  is  simply  a  function  of  k  unspecified  as  of  yet,  and  we  define  ?7(0)  =  1.  Then  (10.15) 
becomes 

I!"’"  S 

for  some  >  0.  Recall  now  that  the  columns  of  Mk  are  just  the  generalized  eigenvectors  of 
and  thus  may  be  chosen  to  have  any  desired  norm.  If  we  choose  the  Mk  such  that  each  column  of 
Mk  has  unit  norm,  then  it  is  trivial  to  show  that  a{Mk)  <  y/n  for  any  where  n  is  the  dimension 
of  the  state  vector  x.  Using  this  fact  we  may  define  the  constant 


a{Mo) 


(10.19) 


SO  that  (10.18)  may  be  written 


\\x2\\<Ke-”^^o^^\\xo\\+Kr){l)e-^^^^^\\xo\\ 


(10.20) 


Proceeding  in  a  similar  manner  it  is  easily  established  that 


||a;3||  <  Ke-^^o^^\\xo\\  +  Kri{l)e-^^^^^^\xo\\ 

I  |a:o  1 1  +  Kr){l)v{2)e-’^^^^^  |  |xo|  | 


(10.21) 


for  some  positive  constants  mzo,  mzn,  mz\2  and  mzz-  Grouping  terms  of  like  powers  of  r?  we  get 

||a;3||  <  +2ir(9(77)e-’"*i^=||a;o||  +  KO{ri^)e-”^^^^^\\xa\\  (10.22) 

for  some  positive  constant  mzi  •  By  induction  it  can  thus  be  shown  that 

||a;,||  <  Karze-^^^^^\\xz\\+KariO{r])e-”^^^^^\\xo\\ 

+Jfa,.2C>(7?2)e-’"-^^-||a;o||+--- 

+/Ca,(,_i)0(r7<’-i)  )e-’"^(’-  ||a;o  1 1  (10.23) 


for  positive  constants  nirj  and  arj,  where  arj  is  given  by  the  shifted  binomial  coelRcient  formula 

(r-  1)! 


(  1  ^ 

r  -  1 


V  '  } 


jl{r  -j-  1)! 


(10.24) 
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If  we  assume  all  of  the  rj{k)  are  bounded  above  by  a  positive  constant  p,  then  from  (10.23)  we  may 


write 


If  we  now  take 


r~l 

j  1 1  ^  ^\\  ^0 1 1  ^  ^  6 

j=0 


rjir  ~  mmnirj  >  0 


TTlri^r 


we  may  pull  the  exponential  term  out  of  the  sum  in  (10.25)  to  get 


r-l 


Now,  recognizing  that 


\\xr\\<K\\xo\\e-^^^’’Y.arifP 

j=0 


r  — 1 

^arjp^  =  (l  +  pr 
j=o 


is  just  a  special  case  of  the  binomial  formula,  we  have 


KI|<i^||a;o||e-'”’-^’'(l  +  p) 


(10.25) 


(10.26) 


(10.27) 


(10.28) 


(10.29) 


Now,  define 

-  1  1 

Sr=-Y^Sj  =  -Ar  (10.30) 

^  j=0  ^ 

so  that  Sr  is  just  the  average  sampling  interval  size.  Then  (10.29)  becomes 

and  thus  it  can  easily  be  seen  that  for  the  norm  of  x  to  decrease  as  the  number  of  time  steps  r 
increases,  we  must  have 

<  1  (10.32) 

^m-rOr 

Taking  the  natural  log  of  both  sides  of  (10.32)  and  rearranging,  we  get  the  equivalent  condition 

mrSr  >  ln{l  +  p)  (10.33) 

We  now  make  one  further  assumption  on  the  original  system  (10.1).  If  /  is  continuously  differentiable, 
then  in  some  finite  neighborhood  of  the  origin  the  system  dynamics  are  governed  by  the  linearization 
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of  /,  so  that  F{x)  -  |fu=0  for  small  ||r||.  Thus,  near  the  origin  not  only  is  F  Hurwitz  by  our 
original  assumption,  but  it  is  also  constant.  The  linearized  dynamics  x  =  F{0)x  have  some  finite, 
bounded  domain  of  attraction,  D/,  which  is  an  invariant  set  and  for  which  every  x  eVi  tends  to  0 
as  r  00.  Define  Bi  as  the  radius  of  the  largest  ball  in  TZ^  (defined  by  the  vector  2-norm)  that  is 
wholly  contained  in  Vi,  and  define  B2  as  the  radius  of  the  smallest  ball  wholly  containing  .  Then, 
if  (10.33)  is  satisfied,  it  follows  that  any  trajectory  of  (10.1)  is  bounded  and  converges  to  the  origin. 
This  may  be  seen  by  considering  two  cases  corresponding  to  trajectories  which  start  either  inside  or 
outside  of  Vi.  If  \\xq\\  <  Bi  so  that  the  initial  condition  lies  inside  the  domain  of  attraction  of  the 
linearized  dynamics,  then  clearly  \\Xr\\  <  r  since  Vi  is  an  invariant  set,  and  additionally  we 
have  0  as  r  ^  00  by  definition  of  2)/.  If  on  the  other  hand  ||a:o||  >  Si,  the  desired  properties 
follow  because  if  (10.33)  is  satisfied,  every  system  trajectory  that  starts  outside  of  T>i  enters  Vi  after 
a  finite  number  of  time  steps  Z,  which  is  computable  by  setting 


and  solving  for  I  to  get 

“  ln{l  T  p)  -  niiSi 

Boundedness  follows  since  outside  of  T>i^  we  have  from  (10.31)  and  (10.32)  that  for  any  r 


(10.35) 


(10.36) 


while  inside  T>i  we  have  ||a;y||  <  S2,  giving  the  overall  bound 

lla^yll  <  max{i^||a?o||,  S2}  (10.37) 

Convergence  of  a:  to  0  follows  trivially  from  xi  being  in  Vi  after  I  time  steps.  Note  that  in  the  above 
development  we  allowed  variable  sampling  intervals,  and  we  ended  up  using  the  minimum  of  all  the 
rUrj  as  our  exponential  decay  factor.  If  we  instead  fix  6j  =  Sr  constant,  then  the  above  development 
also  holds  for 

1 

TUr  =  “  TUrj  (10.38) 

^  i-0 
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so  that  now  we  are  using  the  average  of  the  which  could  be  significantly  larger  than  their 
minimum,  thus  making  (10.32)  or  equivalently  (10.33)  easier  to  satisfy. 

Now,  it  remains  to  be  seen  under  what  conditions  (10.33)  can  be  satisfied.  Intuitively,  (10.33)  is 
easier  to  satisfy  if  and  Sr  are  large,  and  if  p  is  small.  Indeed,  if  p  =  0  we  simply  need  and 
Sr  greater  than  zero,  so  that  we  recover  the  stability  condition  of  a  linear  time-invariant  system  as 
a  special  case.  For  nonzero  p,  however,  we  have  a  minimum  size  requirement  that  we  must  meet  on 
the  product  mr^r*  For  systems  meeting  the  analyticity  and  diagonalizability  assumptions  on  F,  it 
is  known  [38]  that  each  column  of  M  is  a  linearly  independent  eigenvector  of  F  which  is  analytic 
with  respect  to  t.  Thus,  the  size  of  if]k  and  hence  p  is  directly  proportional  to  the  size  of  Sk^  for 
sufficiently  small  Sk^  Additionally,  we  know  from  Chapter  7  that  if  we  choose  Q  >  0  globally,  then 
we  can  increase  the  size  of  for  the  controllable  modes  by  increasing  the  minimum  eigenvalue  of 
Q.  Thus,  8k  and  Q  can  be  used  directly  to  ensure  satisfaction  of  (10.33),  provided  the  uncontrollable 
modes  also  have  sufficiently  small  eigenvalues.  If  F  is  not  globally  simple,  then  it  may  be  possible 
to  perturb  Q  so  as  to  give  a  globally  simple  F.  If  F  cannot  be  made  simple,  the  above  analysis 
becomes  significantly  more  difficult,  due  to  lack  of  guaranteed  analyticity  of  all  the  eigenvectors  of 
F  [42].  It  still  may  be  possible  to  extend  the  above  analysis,  however,  by  considering  the  fact  that, 
for  sufficiently  fast  sampling  and  sufficiently  large  m^,  divergent  trajectories  can  only  occur  along 
infinite  paths  where  F  is  not  simple.  This  is  because  a  trajectory  encountering  a  finite  number  of 
such  points  can  be  thought  of  as  having  a  larger  premultiplying  constant  AT,  which  does  not  affect 
whether  the  system  converges  or  not,  but  instead  only  affects  time  to  converge.  This  line  of  analysis 
may  be  made  rigorous  by  applying  the  sufficient  conditions  for  convergence  of  an  infinite  product 
[1].  If  no  such  infinite  paths  exist,  or  if  it  can  be  shown  that  closed  loop  trajectories  cannot  follow 
such  paths,  then  exponential  stability  may  still  be  provable.  Experimental  analysis  along  these  lines 
supports  this  conjecture.  We  now  seek  to  extend  the  above  concepts  to  exponential  stability  of  the 
sampled  data  SDRE  nonlinear  regulator  algorithm. 
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10.3  Exponential  Stability  of  SDRE 


We  now  repose  the  above  stability  analysis  in  the  context  of  nonlinear  SDRE  regulation,  showing 
that  the  SDRE  problem  reduces  to  a  perturbed  version  of  the  above  problem.  We  are  thus  able  to 
show  that  basically  the  same  conclusions  given  above  apply,  given  our  standard  assumptions.  We 
wish  to  consider  stability  of  the  closed  loop  system  resulting  from  the  nonlinear  SDRE  regulator, 
given  by 

x{t)  =  f{x{t))  =  F{x{t))x{t)  (10.39) 

where  F{x{tk))  =  Fk  and  for  all  t  e  [4,4+i)  we  have  F{x{t))  =  -h  AFfe(a;(t)).  Under  our 
standard  assumptions  of  global  analyticity,  stabilizability,  and  detectability  of  the  open  loop  system 
matrices,  we  have  F  analytic  with  respect  to  t  and  Fk  Hurwitz  for  all  k.  Using  the  above  expression 
for  F{x{t))  we  have  for  any  t  €  [tk,tk-\-i) 

x{t)  =  Fkx{t)  -h  AFk{x{t))x{t)  (10.40) 

which  can  be  seen  to  be  a  perturbed  version  of  (10.2),  with  the  error  term  AFk{x(t))  representing 
the  effects  of  the  changing  nonlinear  dynamics  over  a  sampling  interval.  Thus,  starting  from  any 
initial  condition  Xq  E  TiF  we  may  write 

xi  =  e^^^^xo f  e^^^^^~'^^AFQ{T)x{r)dT  (10.41) 

Jo 

and  we  now  must  be  concerned  with  the  effects  of  the  integral  error  term  on  the  right  hand  side  of 
(10.41),  since  we  have  already  derived  conditions  under  which  the  norm  of  the  terms  deriving  from 
the  first  term  will  tend  to  zero.  Taking  norms  we  may  write 

ft-i 

||a:i||<coe-™°*°  +  ||  /  e^°^^^-^^AFo{T)x{T)dT\\  (10.42) 

Jo 

If  we  assume  that  we  are  sampling  the  system  fast  enough  to  invoke  dominance  of  the  linear  terms 
of  the  power  series  expansions  of  AF  and  we  may  write 

AFo(r)  -  E(r)  ~  Fo  =  Eq  +  rF'  -  Eq  =  rE'  (10.43) 
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where  Fq  is  the  matrix  of  time  derivatives  of  F{x{t))  evaluated  at  to  =  0,  and 

x{t)  =  a^o  +  tFqXq  (10.44) 

so  that  the  second  term  on  the  right  hand  side  of  (10.42)  gives 

II  f  AFo{r)x{T)dr\\  <  ||  /  FoXodrW 

Jo  Jo 

+  II  r  r^e^^^^^-^^F;,FoXodr\\  (10.45) 

Jo 

=  ||t;0||  +  |k0||  (10.46) 

where  i;0  and  ^^;0  are  vectors  respectively  corresponding  to  the  values  of  the  integrals  of  the  first  and 
second  terms  on  the  right  hand  side  of  (10.45),  obtained  by  applying  the  Mean  Value  Theorem  for 
integrals  [1].  The  ith  elements  of  ?;0  and  wO  are  thus  given  by 

vOi  =  (10.47) 

and 

wOi  =  (10.48) 

where  Tj  is  an  x  n  matrix  with  ith  diagonal  element  equal  to  1  and  all  other  elements  equal  to  0 
{Ti  simply  picks  off  the  fth  element  of  the  vector  it  premultiplies).  The  rosi  G  [0,  ^o]  are  the  times 
at  which  evaluation  of  the  integrands  multiplied  by  the  time  interval  Sq  yields  the  value  of  the  zth 
elements  of  the  integrals,  for  s  —  1,  2  respectively  corresponding  to  the  first  and  second  integrals  on 
the  right  hand  side  of  (10.45).  If  we  again  assume  Fk  is  simple,  we  get 

I|t)0||  <irv^(5oToie-'‘“Oi-roi)^(pj/)||a,^||  (10.49) 

and 

\\w0\\  <  K^oTi2e-'^°^^^~^'^MFoMFo)\\xo\\  (10.50) 

where  ho  >  0  is  the  absolute  value  of  the  maximum  real  part  of  the  eigenvalues  oi  Fq,  K  =  >  0 

as  before,  and  tqi  and  ro2  satisfy 

Toi  =  ar^  max  (10.51) 
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(10.52) 


702  =  GLvg  max  7-02^^ 

Note  that  if  tqs  =  0  then  tosi  =  0  for  all  i  and  the  norm  of  ||'z;0||  (for  5  =  1)  or  \\w0\\  (for  6  =  2)  is 
zero.  We  thus  assume  tqs  >  0  and  since  tqs  <  we  may  write 

||t;0||  <  (10.53) 

and 

\\wQ\\<KV^^e-'^°^^^-^^MKPW\\^o\\  (10.54) 

As  in  our  previous  analysis,  we  expect  the  exponential  decay  present  in  (10.53)  and  (10.54)  to  be 
crucial  to  proving  stability.  We  therefore  want  to  ensure  that  tqs  It  is  easily  verified  (by  Taylor 
series  expansions  for  example)  that  for  (5o  small  enough,  the  functions  TiTe~^^x  and  are 

increasing  functions  of  r  for  a  fixed  Hurwitz  matrix  F  and  fixed  vector  x.  Since  these  functions  are 
strictly  increasing,  by  purely  geometric  arguments  it  is  clear  their  integrals  over  small  cannot  be 
equal  to  the  product  of  and  the  integrand  evaluated  at  the  right  endpoint  of  the  interval.  Thus, 
by  sampling  ‘fast  enough’,  we  can  guarantee  that  —  T05  >  0  so  that  we  may  in  fact  write 

t, -ros=—,s  =  l,2  (10.55) 

'^Os 

for  some  uqs  >  1?  which  combined  with  (10.53),  (10.54),  and  (10.46)  gives 

II  ^\M^^-^)AFo{T)x{r)dr\\  <  K^/^6|e-^^^^^W(F')\\xQ\\ 

Jo 

^Ky/^6le-^^^^^a(F^)a{Fo)\\xo  ||  (10.56) 

where  ho  >  aos  —  ^  >  0.  s  =  1,  2.  Since  gos  <  ho,  we  see  that  these  perturbation  terms  decrease 

'^Os 

the  corresponding  effective  exponential  decay  rate.  The  remainder  of  the  analysis  proceeds  as  when 
AF  was  equal  to  zero  in  the  first  part  of  this  chapter.  We  assume  6k  is  selected  small  enough  to 
guarantee  the  constant  overbounds 

knn)  <  (10.57) 

6la{Fl,)a{Fk)<l3  (10.58) 
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so  that  ?;0  contributes  a  perturbation  of  order  6,  and  iwO  contributes  a  perturbation  of  order  P.  Note 
that  these  bounds  are  only  achievable  if  F  and  its  derivative  are  bounded  above.  Upon  expanding 
terms  and  retaining  only  zeroth  and  first-order  terms  in  the  assumed  constant  8^.  —  obtain  an 
expression  of  the  form 


where  Ki  —  a^/nK  >  0  which,  as  expected,  is  the  same  result  as  in  Section  10.2  plus  a  first-order 
perturbation  term  in  8.  We  thus  now  gain  the  attractiveness  condition 


l^8p 

^arS 


<  1 


(10.60) 


in  addition  to  (10.32).  While  the  8  premultiplying  p  in  the  numerator  of  (10.60)  makes  (10.60)  easier 
to  satisfy  than  (10.32),  we  see  that  we  now  also  have  the  reduced  exponential  decay  factor  in  the 
denominator,  which  makes  (10.60)  harder  to  satisfy.  As  discussed  above,  this  decay  factor  will  be 
smaller  than  the  unperturbed  decay  factor  rur  in  (10.32),  but  will  be  no  smaller  than  one  half  ntr  if 
we  sample  rapidly  enough  to  have  be  effectively  linear  in  r  over  the  sampling  interval.  If  this 
is  the  case  and  (10.32)  is  satisfied,  then  (10.60)  will  also  be  satisfied  provided 


p  <  (1  -  28)18^ 


(10.61) 


which  should  be  easily  satisfied  for  reasonably  small  8,  Although  we  have  performed  only  a  first- 
order  perturbation  analysis,  we  nevertheless  see  that  exponential  stability  of  SDRE  regulators  is 
potentially  achievable  under  the  same  conditions  as  given  in  Section  10.2,  plus  some  additional 
boundedness  conditions  on  F  and  its  derivative. 

Prom  the  analysis  performed  in  this  chapter,  we  see  that  exponential  stability  of  SDRE  regulators 
may  indeed  be  provable.  The  line  of  inquiry  pursued  here  indicates  that  sufficiently  rapid  sampling, 
analyticity  of  the  system  matrices,  and  diagonalizability  of  the  closed  loop  dynamics  matrix  play 
key  roles,  as  well  as  the  standard  assumptions  of  globally  stabilizable  and  detectable  factorizations. 
Difficulty  in  guaranteeing  the  diagonalizability  condition  poses  the  most  serious  challenge  to  proving 


exponential  stability  in  this  way. 


XL  Extensions  to  SDRE  Nonlinear  Hoo  Control 


In  this  chapter  we  extend  the  theory  developed  in  the  preceding  chapters  for  nonlinear  regulation  via 
the  SDRE  method  to  SDRE  nonlinear  Hoo  control.  We  emphasize  in  particular  the  similarities  and 
differences  between  the  required  assumptions  and  corresponding  theoretical  developments.  Prior  to 
these  developments,  however,  we  give  some  basic  theorems  addressing  properties  of  standard  and 
Hoo  Riccati  equations. 


ILl  Standard  Riccati  Theory 

Throughout  this  dissertation,  we  have  relied  on  stabilizable  and  detectable  system  factorizations 
guaranteeing  the  existence  of  positive  semidefinite  stabilizing  solutions  to  algebraic  Riccati  equa¬ 
tions  of  the  form  we  encounter  when  applying  SDRE  nonlinear  regulation.  When  considering  Hoo 
type  Riccati  equations,  we  lose  this  guaranteed  solution  existence  property,  except  under  special 
circumstances.  We  thus  now  give  the  theorems  which  explicitly  state  conditions  for  existence  of 
ARE  solutions,  allowing  us  to  see  how  things  change  when  we  consider  the  SDRE  nonlinear  Hoo 
control  problem.  We  precede  the  theorems  with  some  necessary  terminology. 

Consider  the  ARE 

A^P  +  PA  +  PKP  +  Q=zO  (11.1) 


where  A,  Q,  and  K  are  real  n  x  n  matrices  and  K  and  Q  are  symmetric,  and  the  associated 
Hamiltonian  matrix 

\  A  K  ] 


H  = 


(11.2) 


L  J 

We  are  interested  in  existence  of  stabilizing  solutions  of  (11.1),  where  by  stabilizing  we  mean  A-\-KP 
is  a  Hurwitz  matrix.  Under  certain  conditions  on  H  (namely  the  well-known  stability  and  comple¬ 
mentarity  conditions)  [76],  a  stabilizing  solution  P  to  (11.1)  exists,  which  is  uniquely  determined 
by  W,  so  that  the  mapping  ^  P  is  a  function.  We  will  denote  this  function  by  Ric,  so  that 
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P  —  Ric{H)  if  Ti  possesses  the  above  two  properties,  or  equivalently,  if  H  belongs  to  the  domain 
of  Ric,  which  we  denote  by  dom{Ric).  Thus  we  may  replace  the  rather  verbose  expression,  ‘If  the 
Hamiltonian  matrix  H  possesses  the  stability  and  complementarity  properties,  then  there  exists  a 
stabilizing  solution  P  to  (11.1)’,  with  Tf  H  E  doni{Ric),  then  P  =  Ric{H)\  We  now  give  the  two 
theorems  on  which  the  SDRE  theory  of  the  previous  chapters  of  this  dissertation  has  relied,  taken 
from  [76]. 

Theorem  11.1.1  Suppose  H  E  dom{Ric)  and  P=  Ric{H)^  Then 

i.  P  is  real  symmetric 

ii.  P  satisfies  the  ARE  (11.1) 

Hi.  A-[-  KP  is  stable 

Proof:  See  [76],  Theorem  13.5.  ■ 


Theorem  11.1.2  Suppose  H  has  the  form 


(11.3) 


1^  -H'^H  -A^  J 

Then  H  €  dom(Ric)  iff  {A,  B]  is  stabilizable  and  {H,  A}  has  no  unobservable  modes  on  the  imag¬ 
inary  axis.  Furthermore,  P  -  Ric{H)  >  0,  and  Ker[P)  =  Q  iff  {H,  A}  has  no  stable  unobservable 
modes. 


Proof:  See  [76] ,  Theorem  13.7.  ■ 

Note  that  from  Theorem  11.1.2,  detectability  of  {H,A}  is  sufRcieat  but  not  necessary  for  existence 
of  positive  semidefinite,  stabilizing  solutions  to  (11.1).  Also  note  from  the  theorem  the  assumption 
that  the  (l,2)-block  oiTi  be  negative  semidefinite,  which  implies  the  stabilizing  solution  is  maximal, 
i.e.,  the  stabilizing  solution  P+  is  such  that 

P+-P>0,  VP  (11.4) 
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satisfying  (11-1).  We  note  that  this  negative  semidefiniteness  assumption  on  K  does  not  necessarily 
hold  when  we  consider  the  i?oo  Riccati  equation 

A^P  +  PA  +  P{\gG^ -BB'^)P  +  H^H^0  (11.5) 

T 

and  its  associated  Hamiltonian  matrix 

A  ^GG"^  -  BB'^ 

H=  ^ 

-H'^H  -A'^ 

Indeed,  it  is  precisely  this  lack  of  sign  definiteness  which  prevents  guaranteeing  existence  of  stabilizing 
solutions  to  (11.5),  so  that  additional  assumptions  are  required.  Upon  occasion,  we  inay  have  that 
K  =  {l/j^)GG^  —  BB^  >  0.  In  this  case  the  following  theorem  may  allow  us  to  conclude  existence 
of  stabilizing  solutions. 

Theorem  11.1.3  Define  the  quadratic  matrix  function  of  P 

Q{P)  =  A^P  +  PA  +  PKP  +  Q  (11,7) 

and  assume  that  K  >0  and  that  3  a  symmetric  matrix  P  z=z  P'^  such  that  Q{P)  <  0.  If  {A,  K}  is 
stabilizable,  then  3  a  unique  minimal  solution  P-  to  (11.1).  Furthermore^ 

P-<P,  VP  such  that  Q{P)  <  0  (11,8) 

and  A  +  KP^  has  all  its  eigenvalues  in  the  closed  left-half  plane.  If  Q{P)  <  0,  then  P-  <  P  in 
(IPS),  and  A  +  KP-  is  Hurwitz. 

Proof:  See  [76],  Theorem  13.11.  ■ 

Thus,  when  K  is  positive  semidefinite,  we  seek  minimal,  as  opposed  to  maximal  solutions  to  the 
ARE. 

Prom  these  theorems  we  see  that,  in  general,  we  shall  have  to  assume  existence  of  stabilizing 
solutions  to  (11.5),  although  for  K  either  positive  or  negative  semidefinite,  and  {A,K}  stabilizable, 
we  may  be  guaranteed  existence  of  solutions.  With  this  concept  of  conditional  existence  of  solutions 
for  (11.5)  in  hand,  we  now  revisit  some  of  the  analysis  performed  for  the  regulator  type  Riccati 
equations,  in  the  context  of  Poo  type  Riccati  equations. 


(11.6) 
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1L2  Solution  Properties  of  H^q  Scalar  Analytic  Systems 

In  tkis  section  we  consider  SDRE  nonlinear  iJoo  control  for  the  scalar  (single-state)  analytic  case, 
giving  necessary  and  sufficient  conditions  for  obtaining  (locally)  stabilizing  solutions,  just  as  we  did 
for  the  SDRE  nonlinear  regulator  in  Chapter  5.  As  we  did  then,  we  consider  control  of  input-affine 
nonlinear  dynamical  systems  describable  by  a  single  state  variable,  x.  However,  we  now  modify  the 
system  to  include  a  disturbance  term  affecting  the  state  dynamics,  so  that  we  may  write 

X  =  a{x)  -h  b{x)u  H-  g{x)d,  a(0)  =  0 
h(x) 

,  h{0)  =  0  (11.9) 

u 

where  w  is  a  scalar  control,  d  is  a  scalar  disturbance,  2?  is  a  scalar  penalized  variable,  and  a,  6,  g 
and  h  are  assumed  to  be  analytic  real-valued  scalar  functions  of  x.  The  control  objective  is  to  be 
accomplished  by  using  the  SDRE  nonlinear  Hoo  control  technique: 

i.  Write  (11.9)  in  state-dependent  coefficient  (SDC)  form 

X  =  A{x)x  -f  B{x)u  A  G{x)d 
H{x)x 
u 

ii.  Solve  the  nonlinear  Hoo  SDRE 

A{x)p{x)  H-  p{x)A{x)  +  p{x)K{x)p{x)  -h  H^{x)  =  0  (11.11) 

where  K  =  {l/'y‘^)G^{x) 

iii.  Construct  the  state  feedback  via 

u  ~  -B{x)p{x)x  (11.12) 

Comparing  the  above  with  Chapter  5,  we  see  that  the  only  thing  that  has  changed  is  that  we 
now  have  K  taking  the  place  of  in  the  scalar  SDRE.  We  now  investigate  the  results  of  this 
change  in  the  required  assumptions  and  nature  of  solutions  for  (11.11),  and  the  corresponding  local 


(11.10) 
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stability  analysis  for  the  nominal  unforced  {d  =  0)  closed  loop  system.  In  light  of  the  discussion 
in  Section  11.1,  we  expect  the  sign  definiteness  of  K  to  figure  prominently  in  the  analysis.  Recall 
that  we  assume  analyticity  of  the  system  parameters  a,  6,  and  h  and  seek  additional  conditions 
under  which  the  SDRE  Hqo  control  algorithm  yields  an  analytic  locally  stabilizing  state  feedback. 
We  consider  the  same  four  cases  as  in  Chapter  5,  proceeding  analagously,  except  now  we  have  dK 
replacing  2dB  as  the  smallest  nonzero  power  of  x  multiplying  the  term  in  (11.11),  and  ck  replacing 
as  the  associated  nonzero  coefllcient.  We  note  that  for  B  and  G  assumed  analytic,  will 
always  be  an  even  integer.  In  certain  cases  we  must  consider  additional  subcases  corresponding  to 
different  assumed  configurations  of  K.  We  shall  denote  these  additional  subcases  per  the  following 
notation. 

i.  ds  <  so  that  dK  =  2c?b  and  ck  —  — c| 

ii.  ds  >  do,  so  that  dx  -  ^dg  and  ck  ”  {l/j^)c% 

iii.  ds  =  do,  so  that  dK  =  2dB  =  and  ck  =  (l/7^)c^;  -  c% 

1L2J  Hoo  SDRE  Solutions 

Case  1  (2d/f  <  dK  and  2djj  <  dA)  This  case  is  unchanged  from  Chapter  5.  The  lowest-order  nonzero 
part  of  (11.11)  is 

(11.13) 

which,  since  is  nonzero,  has  no  solution  for  all  x.  Obviously,  there  is  again  no  stabilizing  solution 
in  this  case,  reinforcing  that  we  cannot  penalize  powers  of  x  smaller  than  those  on  which  we  may 
have  some  effect  (either  through  the  control  or  through  the  dynamics  themselves). 

Case  2  {dA  <  2d^)  This  case  is  also  unchanged  from  Case  2  of  Chapter  5.  Since  dp  >  0 
and  dA  <  "^ds,  the  stability  of  the  closed  loop  system  is  unaffected  by  the  control,  and  is  in  fact 
determined  by  dA  and  ca^  Invoking  Lemma  5.2.1  we  conclude  that  any  solution  that  exists  is 
stabilizing  iff  dA  is  even  and  ca  <  0. 
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Case  3  {d^  =  dK) 

Case  3A  (2c?jy  >{//<:)  In  this  case,  dp  —  Q  and  (11.11)  reduces  to 


cp(-circp-2c^y^ +---  =  0  (11.14) 

Thus,  the  two  possible  solutions  to  (11.14)  are  Cp  =  0,  Cp  =  -2caIck-  For  Cp  =  0,  the  leading-order 
control  is  u  =  0,  giving  a  closed  loop  system  of  ic  =  Ax  (to  1st  order).  This  solution  is  stabilizing  iff 
the  open  loop  system  is  stable  {d^  is  even  and  <  0).  For  Cp  =  -~2caIck^  the  closed  loop  system 
becomes  (to  leading  order) 

i.  —  +  cb  ^—cb  — — —  ^1  -{-  (11.15) 

and  we  must  consider  the  particular  form  of  ck  dictated  by  the  three  possibilities  given  above  to 
determine  stability. 

Case  3Ai  This  case  is  the  same  as  for  the  nonlinear  regulator.  The  closed  loop  system  becomes 

X  ~  —cax^^^^  -f.  . . .  (11.16) 

which  is  stable  iff  is  even  and  >  0.  Thus,  stabilizing  solutions  exist  for  this  subcase  iff  dA  is 
even,  which  is  always  the  case  since  dA  =  d^^  and  dj(:  is  an  even  integer. 

Case  3Aii  With  ck  =  (1/7^)<^gj  closed  loop  system  becomes 

which  is  stable  iff  is  even  and  <  0.  But  if  this  is  the  case,  the  open  loop  system  is  also  LAS. 
Thus,  we  have  two  stabilizing  solutions  in  this  case  if  <0,  and  no  stabilizing  solutions  if  >  0. 
Note  that  we  have  K  locally  positive  semidefinite  for  this  subcase,  and  if  ca  <  0,  then  {A,  B}  is 
stabilizable.  From  Theorem  11.1.3  we  conclude  that  p  =  0  is  the  minimal,  locally  stabilizing  solution 
to  (11.11), 

Case  3Aiii  The  closed  loop  system  becomes 
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which  has  the  same  solution  and  stability  properties  as  Case  3Ai  if  ck  —  (l/7^)c^;  —  <  0  and 

the  same  properties  as  Case  3Aii  if  ck  —  (l/7^)c^;  ~  c%  >  0  (with  different  coefficients  multiplying 
of  course). 

We  now  recall  our  discussion  from  Chapter  5  about  the  application  of  this  subcase  to  the  regulator 
problem  when  our  cost  function  to  be  minimized  is  identically  zero  {djj  — ^  oo).  For  LTI  open  loop 
stable  systems,  the  LQR  optimal  control  for  zero  cost  function  is  zero,  and  that  is  also  what  we  see 
for  SDRE  nonlinear  regulation  and  Hoo  control,  if  we  take  the  minimal  solution  in  the  ffoo  setting. 
If  an  open  loop  LTI  system  is  unstable  and  controllable,  then  the  open  loop  unstable  poles  are  moved 
to  their  stable  mirror  images  in  the  left  half  complex  plane  (i.e.,  the  real  parts  of  the  eigenvalues  of 
the  closed  loop  system  are  the  negatives  of  their  open  loop  counterparts).  In  the  nonlinear  regulator 
open  loop  unstable  single  state  case,  we  saw  that  the  closed  loop  dynamics  were  the  negative  of  the 
open  loop  dynamics  {ad  =  —aoi).  In  this  SDRE  nonlinear  Hoo  case,  we  have  a  similar  result,  except 
now  we  have  stabilization  only  when  ck  =  (l/7^)c^;  ~  <  0,  and  ad  —  —kaoi,  where  A;  >  0  is  a 

perturbation  factor  reflecting  the  contribution  of  the  (l/7^)c^t  term  in  the  Riccati  solution. 


Case  3B  {2dH  =  d/c)  In  this  case  dp  again  equals  zero  and  the  leading-order  (positive)  SDRE 
solution  is  given  by 


Cp  =  -ca/ck  +  ~  (11.19) 

Substituting  (11.19)  into  u  =  —hpx  and  then  into  (11.9)  we  obtain  the  (leading-order)  closed  loop 
dynamics 


X  = 


(11.20) 


We  only  consider  the  positive  solution  because  it  is  trivial  to  show  that  the  negative  square  root  solu¬ 
tion  always  yields  an  unstable  closed  loop  system.  We  now  again  consider  three  subcases  depending 
on  the  form  of 

Case  3Bi  This  is  the  same  as  the  nonlinear  regulator.  We  get  the  closed  loop  leading-order 
dynamics 

X  =  -^cl  +  {CBCHyx‘‘^  +  ^  (11.21) 
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which  is  always  stable  since  dA  —  is  even. 


Case  3Bii  The  leading-order  nonzero  coefficient  of  p  becomes 

Cp  =  -i‘^caIc%  +  (11.22) 


so  that  we  must  have 


7  4  -  cj^cg  >  0 


(11.23) 


for  a  real  solution  to  exist.  If  (11.23)  is  satisfied,  we  obtain  the  closed  loop  dynamics 


X  = 


ca+7^4 


■Jcg  {ca  -  \/4-44/7^) 


(11.24) 


which  is  stable  if  <  0  and  unstable  if  ca  >  0.  Thus,  a  stabilizing  solution  can  exist  for  this 
subcase  iff  the  open  loop  dynamics  are  LAS,  but  the  locally  stabilizing  solution  is  not  p  =  0  as  in 
Case  3A.  Note  that  stronger  state  penalties  and  disturbance  input  effects  (larger  and  c^)  make 
(11.23)  harder  to  satisfy,  but  is  user-selectable  and  may  be  chosen  to  ensure  existence  of  locally 
stabilizing  solutions  for  some  range  of  7. 

Case  3Eiii  This  subcase  is  like  subcase  3Aiii  in  that  it  is  characterized  by  the  sign  of  ck  = 
(l/7^)cg  -  Cg.  If  Ck  <  0,  then  the  situation  is  like  Case  3Bi,  and  stabilizing  solutions  always 
exist,  given  by  (11.19)  and  (11.20).  If  ck  >  0,  then  the  situation  is  like  Case  3Bii,  in  that  a  locally 
stabilizing,  analytic  solution  exists  iff 

c\-clcK>0  (11.25) 


and  Ca  <  0. 


Case  4  {dA  >  dpc) 

Case  4A  {2dB  =  djc)  In  this  case  dp  =  0  and  (11.11)  becomes 

(cl  +  CKcl)x‘‘^^  +  ---  =  0  (11.26) 


giving 


4  = 


(11.27) 
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so  that  we  must  have  ck  <  0  for  a  real  solution  to  exist.  If  this  is  the  case,  the  closed  loop  dynamics 
become 

x  =  <^s\ch\/ckx‘^‘^^^  +  ---  (11-28) 

which  is  always  stabilizing  since  dfc  is  even  and  ck  <  0. 

Case  4B  (2d/f  >  In  this  case  dp  >  1  and  the  existence  of  solutions  is  highly  dependent  on 
the  particular  structures  of  ^4,  K,  and  H.  We  thus  give  no  criterion  for  the  existence  of  stabilizing 
solutions  in  this  case.  We  also  note  that  such  a  case  is  somewhat  unusual,  in  that  we  would  be 
penalizing  only  large  powers  of  x  without  penalizing  the  smaller,  affectable  powers. 

11,2.2  Discussion 

Prom  the  above  analysis  we  see  that,  as  predicted,  the  sign  definiteness  of  ck  = 
plays  a  crucial  role  in  determining  when  stabilizing  solutions  to  the  scalar  ifoo  Riccati  equation 
exist.  When  ck  <  0,  the  situation  is  analagous  to  the  SDRE  nonlinear  regulator  results  of  Chapter 
5.  This  is  true  because  when  ck  <  0,  controllability  and  stabilizability  are  equivalent  for  the  K 
terms  ((l,2)-blocks  of  H)  of  the  regulator  and  iJoo  Riccati  equations.  Of  course,  observability  and 
detectability  are  always  equivalent  for  the  two  Riccati  equations  since  they  consider  the  same  pair 
{H^A},  Thus,  in  the  scalar  case,  if  we  have  {^(x),  5(a:)}  stabilizable  and  {I{{x),A{x)]  detectable 
near  the  origin,  the  nonlinear  Hqq  control  problem  will  be  solvable  for  some  (potentially  large)  value 
of  7,  since  we  can  guarantee  c/c  <  0  by  choosing  7  large  enough.  This  will  not  be  true  in  general 
when  we  consider  the  multistate  case,  since  directionality  issues,  and  not  just  magnitude  issues,  will 
play  a  role  in  determining  sign  definiteness  of  K.  We  also  saw  that  il  cj^  >  0,  in  some  situations  we 
were  able  to  find  LAS  solutions,  as  predicted  by  Theorem  11.1.3.  It  remains  to  be  seen  whether  these 
additional  conditions  can  be  uniformly  identified  and  exploited.  The  results  of  this  section  again 
verify  the  known  result  of  analytic  and  stabilizable/detectable  systems  yielding  analytic  stabilizing 
solutions  to  (11.11),  but  also  show,  just  as  in  the  regulator  case,  that  stabilizability /detectability  for 
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all  X  are  not  necessary  in  the  scalar  case,  and  illustrate  under  what  conditions  such  solutions  may 
or  may  not  be  obtained. 


1L2.3  Examples 

We  now  present  three  examples  to  illustrate  the  above  theory,  and  to  demonstrate  the  necessary  and 
sufficient  conditions  for  analytic  stabilizing  solutions  derived  herein. 


Example  1 


X  =  ^xu  ~  xd;  h  —  cx^ 


(11.29) 


This  example  is  not  controllable/observable,  nor  stabilizable/detectable  in  a  neighborhood  of  the 
origin.  Here  we  have  A  =  B  —  5a:,  G  =  —a;,  and  H  ~  cx  that  dA  =  2,  —  1,  dn  =  I5  es  = 

5,  do  =  1,  cq  =  -1,  dn  =  1,  and  ch  =  c  and  Case  3Biii  applies.  Since  =  2  is  even,  we  expect 
stability  of  the  closed  loop  to  depend  on  the  sign  of  ck  =  (1/7^)  “  25.  For  7^  >  1/25,  c/c  <  0  and  we 
expect  stable  closed  loop  solutions.  Solving  (11.19)  under  this  assumption  (letting  Ck  —  -A:,  A:  >  0) 
we  find 

Cp  =  1/k  -h  x/l/F  +  c2/"fe  (11.30) 


The  resulting  closed  loop  system  is 


a:  =  [1  —  25(1/A:  +  ^/T/k^~T'^Jk)]x^  H - 


(11.31) 


which  is  indeed  (locally)  stable  since  k  <  25.  If  ca:  >0  we  need  <  0  and  c\  —  c^ck  =  1  —  c^ck  >  0 
for  existence  of  an  LAS  solution.  Since  =  1  >  0,  no  stabilizing  solutions  exist  for  7^  values  smaller 
than  1/25. 


Example  2 


X  =  ~x^  +  ^xu  —  xd;  h  =  cx^ 


(11.32) 


All  we  have  done  here  is  to  negate  A  from  Example  1,  so  that  now  ca  =  ~1  <  0.  Case  3Biii  still 
applies.  For  7^  >  1/25,  ck  <  ^  and  we  expect  stable  closed  loop  solutions.  Solving  (11.19)  as  in 
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Example  1  we  find 


Cp  =  —1/k  /k  (11.33) 

and  the  resulting  closed  loop  system 

i  =  [-1  -  25(-l/fe  +  /l/P  +  cV^)]a:^  +  •  •  •  (11.34) 

which  is  indeed  LAS  since  Cp  =  ~\/k  y/l/k^  +  c^/k  >  0,  Now,  for  Cfc  >  0,  since  ca  <  0,  we 
expect  LAS  solutions  if  1  —  c^ck  >  0,  or  equivalently,  if  ck  =  I/t^  —  25  <  1/c^.  Thus,  in  this  case 
we  can  achieve  LAS  closed  loop  systems  with  slightly  better  disturbance  attenuation  properties  than 
in  Example  1,  because  now  we  get  LAS  solutions  for  y  such  that  25c^-\-i  ^ 


Example  3 


+  bxu  —  xd;  h  =  cx^ 


(11.35) 


Notice  that  all  we  have  changed  from  Example  2  is  a  from  —x^  to  ~x^  so  that  =  1  and  dK  = 
2dB  =  2  so  that  dA  <  2dB  and  Case  2  applies.  Since  ==  1  is  odd,  we  expect  an  unstable  closed 
loop  solution.  Solving  (11.11)  we  find  dp  =  1  and  Cp  =  c^/2,  so  that  p  is  not  even  locally  positive 
semidefinite.  The  resulting  low-order  control  is 

-5C^  q 


u  =  —bx( — x)x  = 

^  2  2 


(11.36) 


giving  the  closed  loop  system 

X  = -x^ - ^x^  A -  (11.37) 

which  as  expected  has  stability  properties  unaffected  by  the  control,  and  is  clearly  unstable. 

From  these  examples  we  see  the  validity  of  the  theory,  and  the  important  role  ck  and  now 
play  in  local  closed  loop  stability  analysis.  Indeed,  this  role  of  K  as  opposed  to  B  in  the  SDRE  is 
the  main  difference  we  observe  between  the  regulator  and  the  Hoo  theory,  whereas  the  fact  remains 
in  both  methods  that  we  cannot  penalize  powers  of  x  smaller  than  we  can  effect,  nor  can  we  stabilize 
systems  with  only  terms  in  B{x)  of  order  higher  than  A{x). 
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11,3  Lyapunov  Stability  for  Systems  with  Full  Rank,  Constant  B  Matrices 

In  this  section  we  are  able  to  show  that  the  global  asymptotic  stability  properties  of  SDRE  nonlinear 
regulators  for  systems  with  full  rank,  constant  B  matrices  carries  over  to  the  SDRE  nonlinear  Hqo 
unforced  {d  =  0)  closed  loop  system,  with  only  two  additional  assumptions.  The  method  of  proof  is 
the  same  as  in  Chapter  7,  and  is  only  slightly  more  complicated.  Recall  we  consider  the  system 

X  =  a(x)  -H  Rrt  H-  Gd,  a{0)  —  0 
h(x) 

z=  ^h{0)=0  (11.38) 

Ru 

where  x,  u  e  77-^,  d  e  TV',  z  G  TV^,  and  R  and  B  are  nonsingular  matrices.  The  vector  functions  a 
and  h  are  assumed  to  be  real-valued  functions  of  x,  so  that  well-defined  global  SDC  parametriza- 
tions  are  guaranteed  to  exist.  The  first  assumption  we  make  is  implied  in  (11.38),  namely  that  the 
disturbances  enter  the  dynamics  through  a  constant  matrix  G.  The  second  assumption  we  make  is 
that  7  is  selected  in  the  suboptimal  Hoo  control  problem  large  enough  so  that 

K  =  (l/y^)GG'^  -  BR-^B'^  <  0  (11.39) 

i.e.,  the  (l,2)-block  of  the  Hoo  Hamiltonian  is  negative  definite,  where  R  =  R  >  0.  Note  that  a 
finite  value  of  7  satisfying  (11.39)  is  guaranteed  to  exist  by  the  positive  definiteness  of  BR~^BF' . 
We  now  formally  state  and  prove  the  result. 

Theorem  11.3.1  Consider  the  system  (11.38)  with  a{x)  and  h{x)  assumed  to  be  functions 
and  G  a  constant  matrix.  Assume  further  that  a{x)  =  A{x)x  =  0  a;  —  0,  rank{B)  —  n  and 
H^{x)H{x)  >  0  V  X  where  h{x)  ==  H{x)x.  Also,  assume  R  in  (11.38)  is  constant  and  nonsingular, 
and  (11.39)  holds.  Then  application  of  the  SDRE  nonlinear  Hqq  control  algorithm  defined  by 

u—  -R~^B^  P{x)x  (11,40) 

where  P{x)  is  the  stabilizing  solution  to 

A^{x)P{x)  +  P{x)A{x)  +  P{x)KP{x)  +  Q{x)  =  0  (11.41) 
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and  K  is  as  in  (11,39),  to  (11.38)  yields  a  closed  loop  system  which  is  globally  asymptotically  stable. 


Proof:  Since  (1L39)  holds  and  H{x)  is  globally  full  rank,  we  note  first  of  all  that  stabilizing 

solutions  to  (11.41)  are  guaranteed  to  exist  for  all  x  by  Theorem  11.1.2,  so  that  the  control  algorithm 
is  globally  well-defined.  The  negative  definiteness  of  K  also  lets  us  write 

K  -DD^  (11.42) 

for  some  nonsingular  matrix  D.  With  this  definition  (11.41)  becomes 

A^{x)P{x)  +  P(x)A{x)  -  P{x)DD'^P{x)  +  Q{x)  =  0  (11.43) 

Consider  the  globally  positive  definite,  decrescent,  and  radially  unbound  Lyapunov  function  V  = 
(1/2)  where 

q  =  D-^x  (11.44) 

Then  we  have  V  =  q.  Differentiating  (11.44)  and  using  (11.40)  we  find 

q  =  =  D-^A{x)Dq  -  BR-^ P{x)Dq  (11.45) 

Similar  to  Chapter  7,  define 

A{q)  =  D-^A{x)D 
P{q)  =  D^P{x)D 

Q{q)  =  D^Q{x)D  (11.46) 

Using  (11.46)  and  (11.45),  V  becomes 

V  =  q^  [A{q)  -  D- 1 BR- ^ P(g)]g  (11 .47) 

so  that 

V  <{(^2[Aiq)]-X[D-^BR-^B'^D-^P{q)])q^q  (11.48) 


11-13 


where  //2(^)  =  max\Re\(^A-\-A^)l2]  is  the  matrix  measure  oi  A  with  respect  to  the  Euclidean  norm. 
We  note  that  (11.48)  is  different  from  the  corresponding  expression  for  V  in  the  regulator  case  only 
by  the  presence  of  the  D~^ BR~^ D~'^  premultiplying  P{q)^  Now,  using  (11.46),  (11.43)  becomes 

A^{q)P{q)  +  P{q)A{q)  -  P{q)P{q)  +  Q{q)  -  0  (11.49) 

SO  that  using  Mori’s  lower  bound  [49]  for  A[^(?)])  we  find  A[P(g')]  >  ^2[A{q)]  or  equivalently 
//2[^(?)]  —  A[-P(q^)]  <  Oj  just  as  in  the  regulator  case,  where  we  recall  we  may  use  Mori’s  bound 
because  A{q)  is  guaranteed  nonsingular  by  assumed  global  nonsingularity  of  A{x)  and  (11.46).  Now, 
since  a{WX)  >  gi{W)a{X)  for  any  two  matrices  W  and  X  [57],  then  —\[D~^BR~^B'^D~'^P{q)  < 
^X[D-^BR-^B'^ D-'^]X[P{q)l  Thus,  if  X[D'^ BR-^^ B^ D-'^]  >  1,  then  by  (11.48)  we  will  have 
V  <  0  for  all  g,  and  the  theorem  will  be  proven.  Now,  recall  that  by  definition  K  =  — 

BR'^^B^  =  —DD^j  so  that  we  may  write 

BR-^B'^  =  DD^  -f  (1/72)GG^ 

Thus,  we  have 

X[D-^BR-^B^D-'^]  ^  X[D-\DD'^  +  (l/7^)GG^)p-'^] 

=  A[/  +  (1/7^)D-^GG^P^^] 

==  1  +  {lh^)X[D-^G{D-^Gf]  >  1  (11.50) 

since  the  minimum  eigenvalue  of  the  positive  semidefinite  matrix  D~^G{D^^GY'  is  greater  than  or 
equal  to  zero,  and  the  theorem  is  proven.  ■ 

We  conclude  this  section  by  noting  that  the  other  theorems  in  Chapter  7  may  be  similarly 
extended  to  the  SDRE  nonlinear  Poo  case  by  making  the  assumption  (11.39),  since  those  theorems 
also  establish  conditions  under  which  ii2[A{q)]  -  X[P{q)]  <  0,  and  (11.50)  holds.  Thus,  all  the 
results  of  Chapter  7  apply  to  the  SDRE  nonlinear  Poo  control  problem,  provided  we  have  negative 
definiteness  of  the  constant  matrix  K  ~  (1/7^)GG^  —  BR~^B^ . 
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114  Asymptotic  Stability  of  Sampled  Data  SDRE  Nonlinear  H^o  Controllers 

In  this  section  we  give  the  changes  in  assumptions  necessary  to  extend  the  results  of  Chapters  8 
and  9  to  the  closed  loop  system  obtained  by  applying  the  sampled  data  SDRE  nonlinear  Hqo  control 
algorithm.  We  also  outline  some  slight  additional  considerations  needed  to  complete  the  proofs.  Our 
control  algorithm  is  determined  analagously  to  the  sampled  data  SDRE  nonlinear  regulator.  That 
is,  at  sampling  time  tk  we  solve  the  sampled  data  nonlinear  JTqo  SDRE 

AjPf,  +  Pf,Ak  ^  PkKkPk  +  Qfc  -  0  (11.51) 

where  Ak  —  A{x{tk))  and  likewise  for  all  other  variables,  Q  =  H^H,  3irLdK{x)  = 

B{x)R~^{x)B'^  (x)  as  in  the  previous  sections  of  this  chapter,  and  apply  the  constant  control 

-  -R-^B'^PkXk  (11.52) 

until  the  next  sampling  time  tk^i .  With  H  selected  globally  nonsingular,  nonlinear  Hqo  versions  of 
Theorems  8,3.1  and  8.4.1  apply  provided  the  following  three  additional  assumptions  are  made. 

i.  G(x)  is  globally  analytic  with  respect  to  x 

ii.  The  pair  {yl(a?),iir(a;)}  is  globally  stabilizable 

iii.  Stabilizing  solutions  to  (11.51)  exist  everywhere  along  the  trajectory 

Assumption  i  is  a  new  assumption,  which  is  sufficient  to  ensure  K{x)  is  globally  analytic  with 
respect  to  x.  Assumption  ii  is  in  addition  to  the  previous  assumption  of  global  stabilizability  of 
{A{x),B{x)],  which  is  no  longer  sufficient  to  guarantee  that  solutions  of  the  Hoo  Riccati  equation 
will  be  globally  analytic  with  respect  to  x.  Thus,  Assumptions  i  and  ii,  along  with  the  other 
assumptions  in  Theorem  8.3.1,  combine  to  ensure  the  global  analyticity  of  P,  which  is  crucial  to 
the  proofs  of  both  Theorems  8.3.1  and  8.4.1.  At  this  point  we  remark  that  Assumption  i  is  no 
more  restrictive  than  the  assumption  that  the  other  system  parameters  are  globally  analytic,  but 
Assumption  ii  could  be  potentially  difficult  to  satisfy,  particularly  if  the  disturbance  inputs  enter  the 
system  in  directions  which  are  not  controllable.  If  ,  on  the  other  hand,  at  each  x  we  have  G{x)  is 
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contained  in  span(5(x)),  then  stabilizability  of  {A,  B}  guarantees  stabilizability  of  {A,K]  for  some 
range  of  7  values.  As  mentioned  throughout  this  chapter,  we  must  be  concerned  with  existence  of 
solutions  to  (11.51),  since  stabilizability  of  {A,B}  and  {A,  AT}  and  detectability  of  {H,  A},  which 
we  assume  hold,  are  not  by  themselves  enough  to  guarantee  that  stabilizing  solutions  to  (11.51) 
exist.  Thus,  we  need  Assumption  hi  to  guarantee  well-posedness  of  the  control  algorithm  along  the 
trajectory.  Of  course,  in  view  of  Assumption  ii  and  Theorem  11.1,2,  Assumption  iii  can  be  satisfied 
by  assuming  that  K  <  0  everywhere  along  the  trajectory,  although  this  can  be  a  quite  restrictive 
assumption.  With  these  modified  assumptions  the  proof  of  Theorem  8.3.1  remains  unchanged,  and 
the  proof  of  Theorem  8.4.1  has  only  a  slight  modification  in  the  computation  of  AH,  which  is 
dictated  by  the  change  in  form  of  the  Riccati  equation.  Instead  of 


AVfe  <  Sk+iSkxJ 
—  Sk+l^kXk 


e/  -f-  Pk  "I"  PkFk 


el  ~  Qk  ~  PkPkRk  ^P^Pk 


Xk  xlO{S‘^)xk 

Xk  +  xlO{S^)xk 


(11.53) 


as  in  the  SDRE  nonlinear  regulator,  (11.51)  rearranges  to  give 


F^Pk  +  PkFk  =  -Qk  -  PkBkRk^BlPk  -  {lh^)PkGkGlPk  (11.54) 


which  is  still  negative  definite  since  Qk  <  0,  Thus,  we  get 


AH  ^  ^k+l^kXk 


el  ~  Qk  ~  PkBkRrB^Pk  -  {imPkGkGlPk 


Xk  -f- XkO{S^)xk  (11.55) 


for  the  SDRE  nonlinear  Hqo  case,  so  that  the  arbitrary  tolerance  e  being  selected  to  be  less  than 
A[Qfc]  is  still  sufficient  to  guarantee  negativity  of  (11.55).  Indeed,  the  extra  term  ~{l/j‘^)PkGkG'^Pk 
in  (11.55)  can  only  help  to  make  AT4  more  negative. 

If  we  now  consider  the  results  of  Chapter  9,  we  see  that  they,  too,  remain  unchanged,  provided 
the  above  three  additional  assumptions  are  made.  We  could,  if  we  so  desired,  redefine  the  Lyapunov 
function  scaling  factor  Sr  for  U  G  (4,  according  to 

S,  =  (if  xlP'kXk  >  xl[Qk  PPkBuRu^BlPk  +  {lh‘^)PkGkGlPk]xk)  (11.56) 

Xk  -^rXk 

=  Sk  (otherwise)  (11.57) 
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where  is  as  before  and  Sq  =  1,  to  take  advantage  of  the  additional  negative  contribution  of  the 
-(l/j‘^)PkGkG^Pk  term  to  (11.55),  but  there  really  is  no  need  to  do  so.  If  we  leave  Sr  defined  as 
in  Chapter  9,  then  although  the  definition  of  E  would  change  to 

E={xe  Tl^\  x'^lQ  +  PBR~^B^P  -h  {l/j^)PGG'^P]x  -  0  or  s{x)  =  0}  (11.58) 

E  remains  the  same,  since  P(x)x  =  0  causes  both  the  second  and  third  terms  in  the  first  expression 
in  (11.58)  to  vanish,  so  that  the  theorems  of  Chapter  9  would  still  apply  to  the  d  =  0  nonlinear 
Hoo  case,  provided  the  three  additional  assumptions  above  are  made.  We  conclude  this  section  by 
noting  that,  for  the  results  of  Section  9.3.6  to  be  valid  in  the  nonlinear  Hoo  setting,  we  must  have 
both  {A^B}  and  {A, if}  globally  stabilizable,  and  that  is  why  we  stated  above  that  Assumption  ii 
was  in  addition  to,  and  not  instead  of,  having  {A^B}  globally  stabilizable. 

11.5  Exponential  Stability  of  Nonlinear  SDRE  Hoo  Systems 

Since  the  arguments  of  Chapter  10  rely  on  stability  and  analyticity  of  the  matrix  function  F  = 
A  -h  BR^^B'^P,  the  assumptions  of  Section  11.4,  which  guarantee  those  properties  for  the  d  =  0 
SDRE  nonlinear  Hoo  F  matrix  function,  are  also  sufficient  to  extend  those  exponential  stability 
arguments  to  the  Hoo  case.  All  other  assumptions  made  in  the  case  of  the  nonlinear  regulator,  such 
as  diagonaiizability  of  F,  would  also  need  to  hold,  of  course.  The  only  other  consideration  that  is 
different  in  the  Hoo  case  is  that  the  use  of  lower  bounds  on  X[P]  to  make  the  exponential  decay  factor 
nir  larger  by  increasing  A [<5]  require  the  additional  assumption  that  K  —  {l/'y^)GG^ —BR~^B^  <  0. 
We  note  that  this  assumption  might  already  be  made  in  order  to  satisfy  Assumption  iii  of  Section 
11.4,  and  thus  may  represent  no  additional  requirement. 

In  this  chapter  we  have  shown  that  the  theory  previously  developed  for  the  SDRE  nonlinear 
regulator  may  be  extended  to  the  nonlinear  Hqq  setting,  provided  suitable  additional  assumptions 
on  the  system  parameters  are  made.  Although  some  of  these  assumptions  can  be  quite  restrictive, 
they  are  no  more  so  than  in  the  corresponding  linear  case,  and  at  least  give  a  sufficient  set  of 
conditions  which  guarantee  well-posedness  of  the  algorithm. 
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XII.  Design  Problem 


In  this  chapter  we  examine  an  Air-Force- relevant  nonlinear  control  problem  of  nontrivial  state  di¬ 
mension  that  we  wish  to  solve  by  the  methods  of  Chapter  2.  The  chosen  problem  involves  momentum 
control  of  an  axial  gyrostat,  with  various  assumed  configurations.  As  discussed  in  Section  1.2,  the 
chosen  problem  exhibits  highly  nonlinear  dynamics  and  limited  controllability.  It  is  therefore  a 
good  test  of  the  applicability  of  modern  nonlinear  control  design  techniques  to  somewhat  realistic 
problems. 

12 A  Problem  Description 

One  important  satellite  attitude  stabilization  technique  is  dual-spin  stabilization.  A  dual-spin  satel¬ 
lite  consists  of  two  bodies  capable  of  relative  rotation,  with  one  body  spinning  relatively  fast  (the 
rotor)  to  provide  stabilization,  and  one  body  (the  platform)  spinning  relatively  slowly  in  order  to 
perform  mission  requirements  (i.e.,  to  remain  earth-pointing).  Typical  deployment  scenarios  result 
in  both  bodies  initially  spinning  at  nearly  the  same  rate  about  a  single  axis  (the  so-called  all-spun 
condition),  so  that  some  type  of  spinup  maneuver  is  required  to  despin  the  platform.  This  spinup  ma¬ 
neuver  is  described  by  strongly  nonlinear  equations  of  motion,  which  are  complicated  by  unbalance 
or  asymmetry  of  either  the  platform  or  the  rotor,  the  existence  of  which  leads  to  several  interesting 
and  often  undesirable  phenomena.  Spinup  dynamics  of  satellites  under  these  conditions  of  imbal¬ 
ance  or  asymmetry  have  received  significant  attention  in  the  literature,  and  the  reader  is  referred 
to  [23]  for  a  complete  survey.  Two  particular  cases  of  interest  are  commonly  studied,  corresponding 
to  assumptions  of  both  bodies  being  dynamically  balanced,  but  with  one  body  axisymmetric  and 
the  other  asymmetric.  These  spinup  maneuvers  are  typically  performed  by  applying  a  small,  con¬ 
stant  internal  torque  to  the  platform  to  decrease  its  rate  of  rotation,  thereby  transferring  angular 
momentum  to  the  rotor.  Under  this  small,  constant  internal  torque  assumption,  one  particularly 
undesirable  phenomenon  observed  for  the  case  of  axisymmetric  platform  and  asymmetric  rotor  is 
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called  resonance  capture  or  sometimes  precession  phase  lock.  Resonance  capture  happens  when  a 
dual-spin  spacecraft  that  starts  in  the  all-spun  condition  (with  nearly  zero  cone  or  nutation  angle) 
departs  from  that  condition  upon  execution  of  the  despin  maneuver,  creating  a  large  cone  angle. 
Hall  [22]  has  derived  moment  of  inertia  related  conditions  for  when  resonance  capture  can  occur,  and 
has  shown  that  the  phenomenon  is  dependent  on  initial  conditions.  An  interesting  nonlinear  control 
problem  is  therefore  to  try  to  accomplish  the  despin  maneuver  by  use  of  other  than  a  constant 
control  torque,  avoiding  the  phenomenon  of  resonance  capture  in  the  process,  or  at  least  extending 
the  neighborhood  of  initial  conditions  for  which  resonance  capture  is  avoided.  In  the  dual  case  of 
asymmetric  platform  and  axisymmetric  rotor,  the  small  constant  torque  is  used  to  spin  up  the  rotor, 
thereby  reducing  the  angular  momentum  and  hence  angular  velocity  of  the  platform.  For  this  set  of 
assumptions  we  obtain  subcases  corresponding  to  oblate,  prolate,  and  transverse  spinup  maneuvers, 
each  of  which  has  distinguishing  interesting  characteristics.  In  particular,  oblate  and  prolate  spinup 
maneuvers  are  associated  with  satellites  possessing  radically  different  inertia  characteristics  (defined 
in  Section  12.2,  yet  both  start  from  relatively  small  cone  angles  (<  25  degrees).  Transverse  spin 
up,  on  the  other  hand,  represents  a  large  cone  angle  regulation  problem  which  can  be  attempted 
for  either  oblate  or  prolate  spacecraft.  Hall  and  Rand  [25]  have  observed  differing  adverse  behaviors 
for  these  three  maneuvers  under  the  small,  constant  torque  assumption,  which  are  associated  with 
the  number  of  open  loop  separatrix  crossings  each  maneuver  encounters,  where  by  separatrix  we 
mean  a  manifold  which  separates  domains  of  action  of  different  equilibria.  The  oblate,  prolate,  and 
transverse  spinups  encounter  zero,  one,  and  two  such  crossings,  respectively.  Oblate  spinup  has 
no  separatrix  crossing  and  thus  no  adverse  effect  on  the  final  cone  angle  is  seen.  The  prolate  and 
transverse  cases,  however,  have  crossings  which  generally  result  in  larger  final  cone  angles,  and  the 
phenomenon  is  typically  worse  for  the  prolate  case  under  the  small  torque  assumption.  In  this  chap¬ 
ter  we  attack  this  second  set  of  problems,  attempting  the  three  types  of  spinup  maneuvers  described 
by  the  technique  of  SDRE  nonlinear  regulation.  For  comparison  purposes,  we  also  demonstrate 
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how  SDRE  nonlinear  Hoo  control,  feedback  linearization,  and  recursive  backstepping  theory  may  be 
applied  to  this  problem. 

12.2  Equations  of  Motion 

A  simple  model  of  dual-spin  spacecraft  which  we  first  propose  to  use  is  the  axial  gyrostat.  An  axial 
gyrostat  is  a  coupled  rigid  body  system,  in  which  the  relative  rotation  between  bodies  is  constrained 
to  occur  about  an  axis  of  symmetry  for  at  least  one  of  the  bodies.  In  accordance  with  the  above 
discussion  and  to  allow  direct  comparison  of  our  results  with  the  analysis  of  [25],  we  make  the 
following  three  assumptions: 

i.  Both  the  rotor  and  platform  are  dynamically  balanced. 

ii.  The  rotor  is  axisymmetric  and  its  relative  spin  axis  is  parallel  to  a  principal  axis  of  the  gyrostat. 

iii.  The  platform  is  asymmetric. 

The  simplified  satellite  dynamics  for  an  axial  gyrostat  under  the  above  assumptions  consist  of  a 
fourth-order  nonlinear  dynamic  single-input  system,  and  are  given  in  [25].  Before  specifying  them 
here,  we  need  some  definitions.  We  define  a  principal  body  axis  coordinate  frame  located  at  the 
center  of  mass  of  the  gyrostat  by  the  vectors  61,62,63  relative  to  inertial  space,  where  Ci  is  the 
axis  about  which  the  platform  and  rotor  may  have  relative  rotation  (the  spin  axis),  and  62  and  63 
are  principal  axes  orthogonal  to  61  (see  Figure  12.1).  We  define  the  scalar  /x  as  the  axial  angular 
momentum  of  the  rotor  (about  the  61  axis),  and  the  vector  x,  containing  three  elements,  as  the 
total  angular  momentum  of  the  gyrostat  about  the  axes.  An  interesting  system  characteristic  is 
that,  since  there  are  no  external  torques  acting  on  the  gyrostat,  we  have  conservation  of  angular 
momentum.  Thus,  ||a:||  is  a  constant,  and  in  fact  we  make  x  dimensionless  by  scaling  so  that  ||a;||  =  1. 
The  control  input  u  will  be  the  scalar  torque  applied  to  the  rotor.  We  also  define  dimensionless  inertia 
parameters  ij  via 

=  1  -  i  =  1,  2,  3  (12.1) 

Ij 
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Figure  12.1:  Gyrostat  Model  of  Axial  Dual-Spin  Spacecraft 

where  Ip  —  Ii  —  Ig,  Is  is  the  axial  moment  of  inertia  of  the  rotor,  and  Ij  are  the  principal  moments 
of  inertia  of  the  gyrostat.  Using  these  definitions,  the  system  dynamics  for  the  gyrostat  are 

“  {h  -  h)x2X3 

^3  =  -  /^)^2 

(12.2) 

which,  interestingly,  are  invariant  under  the  change  of  variables 

{Xi,X2,X3,i2j3)  {-Xl,X3,-X2,~i3,~i2)  (12.3) 

We  note  that  does  not  appear  in  the  system  equations,  but  that  it  does  impact  the  system  via 
initial  conditions  for  spinup.  This  can  be  seen  from  the  fact  that  in  the  all-spun  condition,  the 
initial  conditions  are  {xi ^ X2 ,  Xz ,  fi)  (l,0,0,ii).  We  also  see  from  (12.1)  that  ii  =  h/h^  so  that  ii 
is  always  positive.  The  relationship  between  the  other  two  inertia  parameters  determines  whether 
the  spacecraft  is  oblate  or  prolate.  For  oblate  spacecraft  we  have  iz  <  ^2  <  0  or  equivalently 


12-4 


Ip  >  h  >  h  >  Oj  whereas  for  prolate  spacecraft  we  have  i2  >  iz  >  0  or  equivalently  h  >  h  >  Ip- 
We  note  that  due  to  the  freedom  in  choosing  and  63,  the  above  assumption  12  >  is  represents 
no  loss  of  generality.  Finally,  the  cone  or  nutation  angle  rj  (the  angle  between  the  ei  axis  and  the 
angular  momentum  vector,  x)  is  given  by  the  relationship 

COST]  -  Xi  rf  =  arccosa^i  (12.4) 


12.3  Design  Objectives 

From  a  controls  perspective,  we  have  three  desired  maneuvers  to  perform,  each  of  which  is  charac¬ 
terized  by  inertia  parameters,  initial  states,  and  desired  final  states.  In  oblate  spinup  we  start  with 
a?!  w  1  and  g  =  so  that  we  have  a  small  initial  cone  angle,  and  we  desire  to  drive  (12.2)  to  the 
final  state  (xi,/x)  ==  (1,1),  since  by  (12,4)  and  the  relationship 

Xi  =  IpiJi  +  g  (12.5) 

this  results  in  a  despun  platform  with  zero  cone  angle  (uji  is  the  angular  velocity  of  the  platform 
about  the  ei  axis).  As  mentioned  above,  the  system  has  been  scaled  so  that  ||n||  =  1.  This  implies 
that  if  we  achieve  the  above  objective,  we  must  have  X2  =  —  0,  so  that  actually  we  desire  to  drive 

the  system  to  (xi,  ^2, 0^3, /i)  —  (1,0,0, 1).  In  prolate  spinup,  we  again  start  with  Xi  1,  fi  =  iiXi, 
and  desire  to  drive  the  system  to  the  final  state  {xi^X2,xz,fx)  =  (1,0,0, 1),  but  we  have  different 
inertia  parameters.  In  the  transverse  spinup  maneuver,  we  start  with  Xi  ^  0,  fi  —  iiXi,  and  we  again 
desire  to  drive  the  state  to  {xi,X2,Xz,^)  =  (1,  0,0,1).  The  difference  between  this  maneuver  and 
the  oblate  and  prolate  spinup  maneuvers  is  thus  the  much  larger  initial  cone  angle  in  the  transverse 
spinup  maneuver.  This  maneuver  represents  recovery  from  a  flat  spin,  for  example,  and  should  most 
seriously  stress  the  control  stategies. 

We  note  that  these  control  problems  do  not  qualify  as  normal  unconstrained  regulation  problems, 
but  are  instead  constrained  nonzero  setpoint  problems.  It  is  expected  that  the  constraint  will 
manifest  itself  as  a  lack  of  controllability  in  the  x  states,  which  may  cause  problems  for  some  of  the 
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methods  of  Chapter  2.  Various  approaches  may  therefore  be  considered  to  attempt  to  circumvent 
this  problem,  some  of  which  we  mention  here. 

By  implicitly  recognizing  the  constraint  ||a;||  =  1  on  the  control  problem,  we  might  consider 
attempting  only  the  regulation  of  {x2,X3)  to  (0, 0),  using  the  constraint  to  thereby  drive  Xi  to  one. 
In  the  nonlinear  regulator  setup,  for  example,  this  would  require  not  penalizing  deviations  of  Xi 
from  zero,  so  that  detectability  of  Xi  becomes  necessary  for  existence  of  a  stabilizing  solution  to  the 
Riccati  equation.  Another  complication  of  this  approach  is  that  even  if  X2  and  Xs  are  successfully 
regulated,  under  the  constraint  xi  remains  free  to  take  on  either  of  the  values  plus  or  minus  one. 

Another  conceptual  approach  might  be  to  change  coordinates  by  defining  —  1,  so  that 

in  the  new  coordinates  we  do  indeed  have  a  true  regulation  problem.  However,  this  option  changes 
the  nature  of  the  constraint  from  ||a;||  =  xl  xl  =  1  to  +  2xi  +  a?!  +  a?!  =  0.  Successful 
regulation  of  xi  still  drives  both  X2  and  x^  to  zero,  however,  as  desired. 

A  third  option  would  be  to  use  the  constraint  to  eliminate  one  state  variable  from  the  dynamics. 
However,  since  the  constraint  only  involves  squares  of  the  state  variables,  this  option  would  require 
taking  square  roots,  the  appropriate  signs  of  which  might  be  difficult  to  determine. 

In  all  of  the  above  approaches,  we  still  have  left  unaddressed  the  nonzero  setpoint  problem  of 
driving  /x  to  one.  This  problem  is  easily  solved  by  a  change  of  coordinates  defined  by  letting  p  =  fi—1. 
This  part  of  the  control  problem  then  reduces  to  regulation  of  i/,  which  fits  readily  into  the  SDRE 
nonlinear  regulator  framework. 

12,4  Open  Loop  System  Analysis 

Before  developing  the  control  design,  we  study  the  open  loop  model  (12.2)  and  its  relevant  properties. 
From  Chapter  9,  we  know  that  open  loop  equilibrium  points  can  be  important  to  closed  loop  stability 
analysis.  We  therefore  observe  that,  for  22  ^  h,  the  only  open  loop  equilibrium  points  that  are 
independent  of  /x  are  a:  =  (d=l,0, 0).  Other  equilibria  exist  for  fixed  values  of  /x,  but  since  ^  is 
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obviously  completely  controllable  and  we  have  a  fixed  desired  final  value  for  it,  we  are  interested 
only  in  the  /i- independent  equilibria  specified  above. 

Now,  from  Section  4.7,  we  know  that  the  system  must  be  nonlinearly  stabilizable  for  any  control 
algorithm  to  work.  We  thus  deepen  our  analysis  by  studying  the  nonlinear  controllability  of  (12.2). 
Performing  the  iterative  procedure  for  determining  the  Control  Lie  Algebra  described  in  Chapter  6, 
it  is  easily  found  (using  Mathematica  [73]  for  instance)  that 

0  0  ^2  2x2X3(2^  -  i2Xi  -  isxi) 

0  ~X3  -xiX2  X3{-2fxxi  +  2i3xl  +  i2X2  ~  ^3^:3) 

A3  -  (12.6) 

0  X2  X1X3  X2{~2fj.Xi  -h  2i2xl  -  i2xl  -f-  hxf) 

10  0  0 

so  that  on  an  open  and  dense  subset  of  77^  the  gradients  of  invariant,  uncontrollable  coordinates  of 
(12.2)  are  given  by 

As"-  =  {y&n^\  2/^ As  =  0}  (12.7) 

From  (12.6)  first  observe  that  fi  is  always  controllable.  Thus,  uncontrollable  states  will  always  lie 
in  the  (xi^X2,xs)  space.  Also,  it  is  trivially  verified  that  for  all  y  G  77^,  y  —  [x\  X2  X3  0]^  belongs 
to  A^.  Thus,  A3  is  rank  three  for  all  rr,  and  there  exists  a  coordinate  in  the  state  space  <j){x)  such 
that  d(f)  =  y^  and  0  is  unaffected  by  u.  Solving  for  0  we  find 

(j)  =  c{x\  4-  0^2  +  xl)  (12.8) 

so  that  we  have  recovered  the  constraint  due  to  conservation  of  angular  momentum,  which  is  un¬ 
changeable  by  any  choice  of  control,  so  that  in  fact  0  =  0.  What  this  means  is  that  all  trajectories 
must  remain  on  the  unit  momentum  sphere  centered  at  (a?!,  X2,X3)  =  (0, 0,0)  (plus  the  controllable  /x 
component  trajectory).  This  uncontrollable  coordinate,  by  itself,  does  not  prevent  us  from  reaching 
the  desired  equilibrium  state  since  it,  of  course,  lies  on  the  sphere,  and  thus,  does  not  imply  that 
the  system  is  not  nonlinearly  stabilizable. 

Continuing  our  examination  of  (12.6),  we  see  that  if  X2  —  X3  ~  0,  then  the  uncontrollable  space 
has  dimension  three,  and,  in  fact,  consists  of  the  {xi^X2^X3)  space.  What  this  means  is  that  once  we 


12-7 


hit  either  of  the  equilibrium  surfaces  (±1, 0, 0,/x),  we  are  stuck  there  forever,  regardless  of  how  we 
select  the  control.  The  implication  here  is  that,  if  we  seek  to  drive  the  system  to  a?i  =  1,  we  must 
avoid  trajectories  passing  through  the  equilibrium  at  xi  —  -1,  and  vice  versa. 

The  above  two  cases  are  clearly  deduced  from  examining  the  determinant  of  A3,  which  is  given 

by 

{fet[A3]  -  X2Xs(x2  -  ir3)(0)  (12.9) 


The  zero  in  (12.9)  comes  from  the  global  one-dimensional  rank  deficiency  of  (12.6),  which  gives  a 
globally  one-dimensional  uncontrollable  space.  For  X2  ~  X3  —  0,  the  loss  of  rank  in  (12.6)  is  three, 
again  giving  a  correspondingly  dimensioned  uncontrollable  space.  From  (12.9)  it  can  be  seen  that, 
for  other  locations  in  the  state  space  (namely  ^2  —  0  or  ijs  =  0,  and  X2  =  2^3),  that  A3  may  also 
lose  rank  of  degree  more  than  one.  However,  there  are  no  other  possible  invariant  trajectories  in  the 
state  space  when  /i  — >  1  for  which  the  dimension  of  A3  remains  constant,  and  this  implies  that  no 
other  invariant,  uncontrollable  manifolds  exist  near  the  desired  equilibrium  point.  To  illustrate  this 
concept,  let  X2  =  0,  2:3  0.  Then  A3  becomes 

0  0  —xl  0 

0  -2:3  0  X3{-2iiXi  H-  2hxj  -  ^32;|) 

A3  =  (12.10) 

0  0  XiXs  0 

10  0  0 

In  this  situation,  A^  =  [o^i  0  2^3  0].  Thus,  the  uncontrollable  space  is  the  circle  2?f+a;§  =  1.  However, 
to  stay  on  this  circle  requires  ±2  —  {hxi  ~  fx)x^  =  0.  But  by  assumption  2^3/0  (or  else  we  return 
to  the  case  already  analyzed  above),  so  that  we  must  have  i^xi  —  fi  ~0  or 


fX  =  i3Xi 


(12.11) 


Now,  by  (12.1)  ^3  is  strictly  less  than  one,  and  so  is  xi  (for  X3  ^  0),  so  that  as  we  drive  the  completely 
controllable  /x  toward  one  we  must  eventually  violate  (12.11)  and  leave  the  X2  =  0  constraint.  A 
similar  analysis  for  the  2:3  0,  X2  7^  0  and  -  2^3  =  0  cases  hold.  From  this  analysis  we  conclude 

that,  although  the  system  (12.2)  is  not  globally  nonlinearly  stabilizable,  we  can  expect  reasonable 
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success  in  driving  the  system  to  the  desired  equilibrium  point  if  our  initial  conditions  are  sufficiently 
far  away  from  the  undesired  equilibrium  point  at  the  opposite  pole  of  the  momentum  sphere. 

We  can  obtain  one  characterization  of  ‘suflSciently  far’  by  algebraically  removing  one  of  the  x 
states,  and  repeating  the  above  analysis  on  the  reduced  system.  To  this  end  we  define  the  coordinate 
transformation  qi  =  0^3,  q2  =  qs  —  0,  with  0  defined  as  in  (12.8)  with  c  =  0.5,  and  eliminate  xi 
as  an  independent  variable  by  writing 


Mq)  =  -9i  -9I 


(12.12) 


Since  the  Jacobian  of  this  mapping  is 


J- 


0  0  1 

0  1  0 

Xi  X2 


(12.13) 


we  see  by  the  Inverse  Function  Theorem  that  the  mapping  is  not  one-to-one  in  a  neighborhood  of 
xi  =  0.  Thus,  we  need  to  know  which  hemisphere  of  the  momentum  sphere  we  are  in  to  complete  the 
mapping,  so  as  to  choose  the  appropriate  sign  in  (12.12).  With  this  coordinate  change  the  equations 
of  motion  become 


=  (/^  - 

h  =  {kxi  -  fi)qi 


fi  =  u 


(12.14) 


where  Xi  is  defined  as  in  (12.12),  and  we  have  eliminated  the  trivial  state  equation  =  0.  The 
nonlinear  controllability  procedure  for  this  system  yields 


0 

q2 

qixi 

A2  == 

0 

-Qi 

q2Xi 

1 

0 

0 

(12.15) 


which  has  determinant 


det[A2]  -  xi{qj  +  g|) 


(12.16) 
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This  distribution  thus  has  full  rank  as  long  as  a'l  ^  0  and  qi  —  and  q2  =  X2  are  not  both  zero. 
Summarizing,  we  see  that  if  we  restrict  x  to  remain  in  a  hemisphere  of  the  momentum  sphere  (either 
a?!  >  0  or  <  0),  then  our  coordinate  change  is  well-defined  (we  know  what  sign  to  take  in  (12.12)), 
and  the  only  place  we  lose  controllability  is  at  the  desired  equilibrium  (^i,  ^^2)  (O5  0)- 

The  final  issue  we  wish  to  explore  in  open  loop  analysis  is  the  existence  of  stabilizable  and 
detectable  factorizations  of  (12.2),  which  we  require  in  order  to  guarantee  well-posedness  of  the 
control  algorithm.  Since  (12.2)  is  globally  analytic  with  respect  to  x,  we  know  that  in  a  neighborhood 
of  the  origin,  the  SDRE  regulator  reverts  to  the  linear  quadratic  regulator  acting  on  the  linearization 
of  (12.2).  A  necessary  condition  for  well-posedness  of  problems  for  numerical  SDRE  methods, 
as  mentioned  in  Section  2.4.3,  is  thus  that  the  system  to  be  controlled  have  a  stabilizable  and 
detectable  linearization.  We  also  know  from  Chapter  6  that,  for  any  SDC  dynamics  parametrization 
a(2:)  ^  A{x)x  of  (12.2),  A(0)  must  equal  the  Jacobian  of  a  evaluated  at  zero  (and  likewise  for  h{x)  — 
so  that  this  necessary  condition  becomes  stabilizability  of  {/(0),^(0)}  and  detectability  of 
{ir(0),  J{0)}.  Computing  the  Jacobian  of  (12.2)  we  find 

0  0  0  0 
0  0  0  0 

J(0)  -  (12.17) 

0  0  0  0 

0  0  0  0 

so  that  the  linearized  controllability  matrix  is  given  by 

0  0  0  0 
0  0  0  0 

Mcf{0)  =  (12.18) 

0  0  0  0 

10  0  0 

The  linearization  is  obviously  not  controllable,  so  that  detectability  requires  left  half  plane  eigenval¬ 
ues  for  all  three  x  states.  From  (12.17)  we  see  that  all  three  of  these  eigenvalues  are  zero,  however. 
The  SDRE  control  is  thus  not  well-defined  for  this  sytstem  in  a  neighborhood  of  the  origin.  Recall, 
however,  that  we  will  never  actually  approach  the  origin  since  all  trajectories  must  lie  on  the  unit 
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momeatum  sphere,  and  that  actually  we  need  to  shift  the  origin  to  make  the  desired  closed  loop 
equilibrium  point  our  origin,  as  discussed  in  Section  12.3.  To  do  so  we  define 

xi  =  X\  —  \  (12.19) 

(12.20) 

transforming  (12.2)  into 

^1  =  (*2  -  H)X2Xz 

X2  =  (hxi  -  v)X2  +  (^3  -  1)2:3 
2:3  =  -(*2^1  -  l^)X2  -  {h  -  1)®2 

v=u  (12.21) 

which  has  Jacobian  at  zero 

0  0  0  0 

0  0  *3-10 

J(0)  =  (12.22) 

0  1  -  *2  0  0 

0  0  0  0 

and  controllability  matrix  the  same  as  (12.18).  Regardless  of  the  values  of  *2  sind  *3,  we  see  that 
(12.22)  always  has  a  zero  eigenvalue  for  a;i,  corresponding  to  the  constraint  manifold  a;! +2:1 +223  =  1. 
Thus,  for  well-posedness  of  the  SDRE  algorithm,  we  need  to  remove  the  221  state  from  the  system 
equations,  and  return  to  a  system  of  the  form  (12.14).  However,  we  still  need  to  incorporate  (12.20), 
giving  the  transformed  system  equations 

i2X\)q2  +q2 

q2  =  (*32:1  -  u)qi  -  qi 

u  =  u  (12.23) 
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which  has  Jacobian  at  zero 


/(0)  = 


and  linearization  controllability  matrix 


Mcf{0)  = 


0  1  0 

-10  0 
0  0  0 

0  0  0 

0  0  0 

1  0  0 


(12.24) 


(12.25) 


From  (12.24)  we  see  that  still  the  linearization  has  eigenvalues  on  the  imaginary  axis,  so  that  the 
requirements  of  stabilizability  are  not  satisfied  near  the  origin,  where  from  now  on  the  origin  is 
the  desired  closed  loop  equilibrium  in  the  transformed  state  space.  As  one  might  expect  from  the 
presence  of  imaginary  axis  eigenvalues  in  (12.24),  application  of  the  SDRE  nonlinear  regulation 
algorithm  to  (12.23)  with  small  initial  cone  angle  and  only  the  u  state  penalized  results  in  u  being 
driven  to  zero,  while  the  qi  and  q2  states  enter  and  remain  in  an  oscillatory  limit  cycle  about  the 
origin.  If  we  penalize  all  three  states,  it  is  interesting  that  all  three  states  end  up  in  a  limit  cycle. 
Typical  simulation  plots  for  these  cases  are  given  in  Section  12.6. 

Remark:  Unfortunately,  this  is  the  best  we  can  do  with  the  system  as  is,  which  points  out  an 

interesting  potential  limitation  of  the  numerical  SDRE  methods.  This  potential  limitation  is  that, 
for  some  systems  with  linearly  uncontrollable  purely  imaginary  open  loop  eigenvalues  in  their  lin¬ 
earizations,  it  may  be  possible  to  stabilize  the  corresponding  modes  with  a  nonlinear  control  law. 
The  numerical  SDRE  methods,  however,  revert  to  standard  linear  design  methods  near  the  origin, 
and  so  will  not  in  general  stabilize  such  systems.  We  conjecture  that  this  limitation  could  possibly  be 
overcome  by  modifying  the  SDRE  algorithm  to  allow  neutrally  stabilizing  solutions  [56]  to  AREs  (the 
closed  loop  F  matrix  has  eigenvalues  with  real  parts  less  than  or  equal  to  zero  as  opposed  to  strictly 
less  than),  and  using  center  manifold  theory  to  prove  stability  along  the  zero  eigenvalue  manifolds. 
This  modification  would  also  require  new  guidelines  for  allowable  choices  of  h,  namely,  penalizing 
only  powers  of  the  state  affectable  through  A  or  B.  This  conjecture  is  based  on  the  analytical  solu- 
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tion  results  of  Chapter  3  and  the  scalar  system  results  of  Chapter  5,  and  has  been  verified  in  a  small 
number  of  simulations. 


12.5  Addition  of  Off-axis  Rotor  for  Stabilizability/ Detectability 

We  saw  in  the  previous  section  that  the  SDRE  regulator  cannot  in  general  achieve  asymptotic 
stability  of  the  desired  equilibrium  point  for  the  baseline  gyrostat.  We  therefore  alter  the  base 
configuration  by  addition  of  a  second,  off-axis  rotor  with  control  law  already  specified  to  achieve 
energy  dissipation  or  damping  in  the  closed  loop  system.  The  effect  of  this  damping  is  achieved  by 
introduction  of  an  additional,  coupled  state  equation  for  the  off-axis  rotor  angular  momentum  about 
its  spin  axis,  which  we  shall  call  Hall  has  developed  and  analyzed  the  resulting  equations  of 
motion  for  the  two  rotor  case  [24] ,  and  we  give  them  here  for  the  specific  choice  of  the  off-axis  rotor 
spin  axis  being  in  purely  the  62  direction.  For  the  reduced  system  with  xi  a  function  of  q  (recall 
(12.12))  we  get 

=  i2Xi)q2  +  ^2  -  Oi2XiP2 

h  = 

P2  =  ^2^2  —  d2P2 

v  =  u  (12.26) 


where 

1 

62  =  02-^52 

=  l  +  (12.27) 
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and  Is^  is  the  ofF-axis  rotor  principal  moment  of  inertia  with  respect  to  the  62  axis.  For  (12.26)  we 
now  find  the  Jacobian  at  zero  to  be 


J(0)  = 


0  1  —  ^2  0 

^3-1  0  0  0 

0  62  — ^2  0 

0  0  0  0 


so  that  the  linearized  controllability  matrix  is  again  given  by 


(12.28) 


0  0  0  0 


Mc/(0)  - 


0  0  0  0 


(12.29) 


0  0  0  0 
10  0  0 

but  now  stabilizability  of  the  linearization  requires  that  the  eigenvalues  of  the  upper  left  3x3 


subblock  of  (12.28)  have  negative  real  parts.  If  one  realizes  the  simple  equalities  1  —  ^2  =  a2  and 
23  —  1  =  — 1//3  =  — CK3,  then  the  characteristic  equation  of  the  said  subblock  is  easily  computed  to 


be 

H"  “h  q;20!3A  -{-  Oi2^3  —  0  (12.30) 

Now,  using  Routh’s  criterion  [17],  we  compute  the  Routhian  array 

1  020:3 

d2  0203 

(12.31) 

(^20:20:3  —  0203  )/d2 

0203 

and  left  half  plane  eigenvalues  of  (12.28)  are  guaranteed  if  all  entries  in  the  first  column  of  (12.31) 
have  the  same  sign.  Since  the  third  element  in  the  first  column  of  (12.31)  simplifies  to  620203/^2 
and  since  0:2,  03,  ^2,  and  €2  are  all  positive,  we  satisfy  Routh’s  criterion  so  that  the  linearization  of 
(12.26)  is  indeed  stabilizable.  We  also  see  from  this  analysis  that  detectability  will  be  guaranteed, 
provided  has  an  independent,  globally  positive  definite  penalty  via  the  SDC  parametrization 
h{x)  =  H(x)x. 
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The  above  discussion  shows  that  adding  the  ofF-axis  damping  rotor  guarantees  stabilizability  and 
detectability  in  some  small  neighborhood  of  the  origin,  for  any  parametrization  A,  and  for  a  suitable 
parametrization  H  which  has  a  positive  definite  term  independently  penalizing  u.  We  are  interested, 
however,  in  performing  transverse  spinup  maneuvers  which  start  far  away  from  the  origin,  so  that 
something  better  than  stabilizability /detectability  of  the  linearization  is  desired.  We  thus  propose  a 
factorization  A  which  guarantees  stabilizability /detectability  everywhere  in  the  positive  momentum 
sphere  {xi  >  0).  We  first  make  some  general  comments  regarding  factorizations.  Although  it  is 
always  possible  to  introduce  fictitious  terms  of  second  and  higher  order  into  the  dynamics  which 
theoretically  cancel,  in  numerical  implementations  it  is  unlikely  that  exact  cancellations  will  result, 
and,  even  if  they  do  cancel,  adding  such  terms  may  not  be  a  good  idea.  To  illustrate,  for  the  two 
state  system 

±1  ~  CL\{x) 

^2  ==  ^2(2;)  (12,32) 


suppose  that  ai  and  a2  are  purely  linear  functions  of  x.  Then  a  natural  choice  for  A{x)  is  the  actual 
matrix 


an  ai2 


(^21  ^^22 

that  results  from  evaluating  the  Jacobian  of  a  at  a;  —  0.  However,  strictly  speaking 


A{x)  = 


(^11  H“  (^1^2  ^12  ~~ 


^21  "b  ^2^2  ^22  ^2^1 


(12.33) 


is  a  valid  parametrization  for  any  Ci  and  C2,  since  the  CiXiX2  terms  cancel  in  A{x)x.  The  addition 
of  these  fictitious  factors,  however,  changes  the  pointwise  appearance  of  the  dynamics  from  the  true 
dynamics  given  by  the  matrix  A.  Thus,  although  such  techniques  have  been  used  for  a  =  0  to  derive 
stable  control  laws  using  the  SDRE  technique  [14],  they  will  not  be  used  here. 

Given  this  consideration,  when  we  consider  (12.26)  and  realize  that  Xi  is  no  longer  considered 
a  state  nor  easily  factored  into  linear  functions  of  the  states  qi  and  q2,  then  we  see  that  the  only 
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terms  we  need  to  decide  how  to  factor  are  the  uq2^  uqi  terras  in  the  ^2  dynamics,  respectively. 
Since  u  is  completely  controllable,  heuristically  we  want  the  factorization  to  show  strong  pointwise 
linear  controllability  of  qi  and  ^2  through  u.  We  thus  propose  the  factorization 

0  1  -  ^22^1  -OL2X1  q2 

i^xi  —  1  0  0  —^1 

0  62  — ^2  0 

0  0  0  0 

This  factorization  has  the  factored  controllability  matrix  function 

0  q2  “^2^1  66^2  +  n2e2a;igi 

0  -^1  ^1^2 

0  0  —62^1  e2(fi^2  +  ^2^1) 

10  0  0 

where 

Cl  ^  -  1 

^2  =  1  —  ^23^1  (12.36) 

The  matrix  function  (12.35)  has  determinant 

det[Mcf]  =  —e2[CiQ2  ^2^1  ^2^1  “  ^i^2^29i  +  (e2a2a;i  —  ^2^2)91]  (12.37) 

which  obviously  loses  rank  for  nontrivial  values  of  qi  and  ^2?  so  that  (12.34)  does  not  yield  a  globally 
controllable  paramerization.  However,  (12.34)  does  yield  guaranteed  stabilizability  for  xi  >  0,  as  we 
now  show  by  Routh’s  criterion  and  some  simple  analysis.  For  stabilizability  of  {A,  5},  recall  that 
we  must  have  [76]  that,  for  all  A  and  y  such  that  y'^A  -  y'^X  and  ReX  >  0,  y'^B  ^  0.  Thus,  we 
consider  the  eigenstructure  of  (12.34).  Setting  the  determinant  of  A{x)  ~  XI  equal  to  zero  we  find 
the  characteristic  equation 

A(A^  +  d2X^  —  +  Ci^s)  =  0  (12.38) 


(12.35) 


(12.34) 
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where  (i,  (2  ^  (12.36)  and 


6  ==  ^262^1  ~  6c^2  (12.39) 

We  thus  have  a  single  zero  eigenvalue,  plus  three  more  eigenvalues  determined  by  the  roots  of 
the  term  in  parentheses  in  (12.38).  For  the  zero  eigenvalue,  we  easily  find  the  corresponding  left 
eigenvector  to  hey^  =  [0  0  0  1].  Since  y^B  —  1  7^  0,  this  zero  eigenvalue  is  stabilizable.  We  now 
show  that  the  remaining  three  eigenvalues  have  negative  real  parts  under  some  slight  additional 
assumptions,  so  that  stabilizability  is  guaranteed.  Recall  that  a  necessary  condition  for  only  left  half 
plane  roots  of  a  polynomial  is  that  all  the  coefficients  have  the  same  sign  [17].  We  thus  must  have 


0 

A 

(12.40) 

-66  >  0 

(12.41) 

66  >  0 

(12.42) 

We  have  (12.40)  satisfied  trivially,  and  since  <  1  and  ^2,  h  <  1?  we  also  have  from  (12.36)  that 
<  0  and  ^2  >  0-  Thus,  (12.41)  is  satisfied,  while  for  satisfaction  of  (12.42)  we  require  ^3  <  0.  We 
must  now  break  up  the  analysis  into  prolate  and  oblate  cases. 

For  the  prolate  case  recall  we  have  ^2  >  ^3  >  0.  Thus,  from  (12.36)  we  see  in  this  case  that,  as 
long  as  cci  >  0,  we  have 

^3  <  Oi2C2  +  (^2  ~  1)^2  (12.43) 

Using  the  definition  of  (2  (12.1)  and  Oi2  (12.27),  (12.43)  becomes 

^3  <  <^2^2  ™  oi2lpd2  —  012(62  “  Ipd2)  (12.44) 

Now,  using  the  definitions  of  ^2  and  62  (12.27),  (12.44)  can  be  written 

6  <02(024,(1 -4) -4)  (12.45) 

Thus,  ^3  <  0  and  vve  satisfy  the  necessary  condition  for  left  half  plane  eigenvalues  regardless  of  the 
values  of  a2  and  4^  if  ip  >  1.  In  the  examples  done  in  this  chapter,  we  have  nondimensionalized  the 
problem  so  that  ip  —  1,  and  thus  we  satisfy  the  necessary  condition.  Note  that  even  if  0  <  ip  <  1, 
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then  the  off-axis  rotor  moment  of  inertia  is  typically  quite  small  compared  to  /p,  so  that  the 
necessary  condition  should  be  satisfied. 

Returning  now  to  the  oblate  case,  recall  we  have  h  <  h  <  0-  Thus,  from  (12.36)  we  see  in  this 
case  that,  as  long  as  xi  >  0,  we  have 

^3  ^  0:262  —  ^2  (12.46) 

Now,  using  the  definitions  of  ^2  and  62  (12.27),  (12.46)  can  be  written 

6<4.c.2(o2-1)-1  (12.47) 


Thus,  ^3  <  0  and  we  satisfy  the  necessary  condition  for  left  half  plane  eigenvalues  in  the  oblate 
case  regardless  of  the  value  of  if  02  <  1.  Even  if  this  is  not  the  case,  the  smallness  of  1 82  is 
usually  sufficient  to  guarantee  ^3  <  0.  For  example,  in  this  chapter  the  simulations  for  the  oblate 
case  assume  Is2  =  0.1  and  a2  =  1.3,  so  that  (12.47)  gives  ^3  <  —0.961. 

Thus,  we  have  necessary  conditions  for  stability  of  the  three  nonzero  eigenvalues  of  (12.34),  and 
we  have  shown  that  we  satisfy  these  conditions  for  the  examples  simulated  in  this  dissertation. 
We  still  need  to  demonstrate  satisfaction  of  a  sufficient  condition  for  stability  of  these  eigenvalues, 
however.  We  now  do  so  by  computing  the  Routhian  array  for  the  parenthetical  term  in  (12.38)  to 
find 


1 

d2  ^1^3 


(12.48) 


(-d266-66)M 

and  observing  that  for  sufficiency  we  thus  need  to  satisfy  the  necessary  conditions  (12.40)  and  (12.42), 
along  with  the  additional  condition 


—(i{d2^2  H-^s)  >  0 


(12.49) 


Since  <  0,  using  the  definition  of  ^3  we  find  that  (12.49)  becomes 


(^2^2  +  0:2^^162  —  ^2^2  —  Ol2Xie2  >  0 


(12.50) 
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which  holds  as  long  as  Xi  >  0,  which  has  been  assumed  throughout.  Thus,  we  have  shown  that 
(12.34)  gives  a  stabilizable  parametrization  for  both  the  oblate  and  prolate  cases,  as  long  as  trajec¬ 
tories  remain  in  the  positive  (a;i)  momentum  sphere. 

Remark:  The  existence  of  this  stabilizable  parametrization  enabling  use  of  the  SDRE  technique 

throughout  the  positive  hemisphere  is  very  interesting,  particularly  for  the  prolate  case,  since  a  man¬ 
ifold  of  open  loop  equilibrium  points  with  unstabilizable  local  linearizations  is  known  to  exist  [24j  in 
the  positive  hemisphere  for  the  prolate  case.  The  implication  here  is  that,  if  a  trajectory  were  to  pass 
through  this  manifold,  then  a  control  algorithm  based  on  Riccati  equations  and  local  linearizations 
such  as  the  LMI  based  techniques  of  [5]  and  [741  noould  fail  to  yield  computable  controllers  at  such 
a  point,  whereas  the  SDRE  methods,  which  use  the  exact  parametrization  (12.34)  of  the  dynamics 
instead  of  a  local  linearization,  still  work  satisfactorily  at  such  points. 


Note  that  the  parametrization  (12.34)  corresponds  to  the  choices  of  ci  =  C2  =  0  for  the  more 
general  possible  factorization 


0  l-i2Xi+Ciiy  ~a2Xi  {l~Ci)q2 


A{x)  = 


izXi  -I  —  C2V 

0 


0  0  -(1  -  C2)qi 

62  — <^2  0 


0 


0  0  0 


(12.51) 


and  we  observe  that,  if  we  pick  ci  =  C2  =  1,  then  the  q  dynamics  appear  to  be  pointwise  unaffectable 
from  the  controllable  variable  u,  so  that  the  factored  controllability  matrix  for  such  a  system  is  equal 
to  (12.29)  for  all  x.  Thus,  the  uncontrollable  space  for  (12.51)  from  a  factorization  standpoint  is 
all  of  the  (91,92,^2)  space,  as  opposed  to  the  much  smaller  subset  of  this  space  determined  by 
setting  (12.37)  equal  to  zero,  and  determining  the  left  nullspace  of  (12.35)  at  the  resulting  points. 
Stabilizability  of  the  factorization  (12.51)  thus  requires  that  all  three  nonzero  eigenvalues  of  (12.51) 
be  stable,  which  was  sufficient  but  far  from  necessary  for  stabilizability  of  the  chosen  parametrization 
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(12.34).  Computing  the  characteristic  equation  of  (12.51)  we  find 


A[A^  -h  —  Ci)(C2  +  +  (^1  —  v)(e20i2^i  ~  <^2(^2  i^))]  —  0  (12.52) 

and  we  see  that  (12.52)  differs  from  (12.38)  only  in  the  first  and  zeroth-order  coefldcients  of  A. 
However,  this  difference  is  significant  since,  if  we  consider  the  prolate  case  with  «  1,  there  exist 
values  of  u  which  render  the  first-order  coefficient  negative.  For  example,  let  ^2  —  0-5  aiid  ^3  —  0.3. 
Then  the  first-order  coefficient  becomes  (//+0.7)(z^+0.5),  so  that  for  all  values  of  z/  €  (-0.5,  -0.7),  the 
coefficient  is  negative,  and  thus  the  factorization  cannot  be  stabilizable.  As  an  additional  comment 
we  mention  that  this  issue  is  not  just  of  theoretical  importance  for  this  problem.  Simulated  attempts 
at  SDRE  nonlinear  regulation  of  the  gyrostat  using  (12.51)  became  numerically  unstable  when  points 
in  the  trajectory  were  encountered  where  the  factorization  was  not  stabilizable.  We  mention  this  case 
in  particular  because  it  corresponds  to  a  natural  SDC  parametrization  that  arises  from  considering 
the  gyrostat  as  a  Hamiltonian  system  as  in  [23].  In  this  framework,  the  nonreduced  equations  of 
motion  for  the  gyrostat  can  be  written 

X  ^  -vn^x 

fi  =  u  (12.53) 

where  V  represents  the  gradient  operator  and  H  is  an  appropriate  Hamiltonian.  Thus,  the  point 
to  be  made  from  all  the  above  analysis  is  that  selection  of  an  appropriate  SDC  factorization  for  a 
nonlinear  system  can  be  nontrivial,  and  pointwise  controllability  issues  should  play  a  strong  role, 
whereas  other  issues  such  as  elegance  of  derivation  of  an  SDC  form  may  be  inconsequential. 

Finally,  we  need  to  consider  detectability  issues.  Recall  that  for  detectability  we  must  have  that 
Ay  =  Xy  and  ReX  >  0  implies  Hy  7^  0.  Now,  since  we  only  have  one  eigenvalue  of  (12.34)  that  has 
ReX  >  0  (A  =  0),  then  the  requirements  for  detectability  are  the  same  as  the  necessary  condition  for 
global  asymptotic  stability  given  in  Chapter  9,  namely  that  if  Ay  ~  0,  then  we  must  have  Hy  ^  0 
for  y  ^  0.  A  sufficient  condition  for  global  detectability  is  clearly  then  to  pick  a  globally  nonsingular 
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iJ,  as  pointed  out  in  Chapter  9.  We  therefore  propose  a  choice  of  H  to  be 


H  —  diag[ki^k2,ks,ki]  (12.54) 

with  ki>  0  for  all  i  =  1,  2,  3,  4.  Also,  as  long  as  k^  /  0,  then  Hy  =  0  implies  u  —  0.  For  z/  =  0, 
then  q2  =  a2  =  0  implies  qi  =  0.  Finally,  setting  ai  =  as  =  0  gives 

a2Xi 

92  =  -/X2  (12.55) 

^2 

which,  unless  a2Xi/^i  —  d2 /es,  are  lines  that  intersect  only  at  q2  =  fi2  =  0.  Thus,  for  nonequality  of 
the  two  slopes  in  (12.55),  Hy  u  ^  0,  and  if  i/  =  0  then  Ay  =  0  only  for  y  —  0,  Substituting  the 
parameter  values  used  later  in  the  simulations,  for  the  oblate  case  we  get  equality  of  the  slopes  for 
xi  =  -6.65,  and  we  get  Xi  —  1.79  for  the  prolate  case.  Since  neither  of  these  xi  values  is  possible, 
taking  ki  =  k2  —  ks^  0  and  ^4  ^  0  in  (12.54)  guarantees  global  detectability  and  also  satisfaction 
of  the  necessary  condition  for  global  asymptotic  stability  of  the  closed  loop  system.  We  investigate 
the  effect  of  using  these  two  choices  for  H  in  the  next  section. 

12.6  SDRE  Nonlinear  Regulator  Simulation  Results 

In  this  section  we  give  typical  simulation  results  for  the  three  spinup  maneuvers  described  in  Section 
12.1  using  the  sampled  data  SDRE  nonlinear  regulator  with  A{x)  and  H{x)  selected  according  to 
(12.34)  and  (12.54),  respectively.  For  comparison  purposes  we  also  include  the  same  maneuvers 
performed  under  the  commonly  made  [23]  small,  constant  torque  assumption.  It  is  shown  that 
the  SDRE  nonlinear  regulator  is  more  effective  in  driving  the  cone  angle  to  zero  than  the  constant 
torque  maneuver,  and  also  offers  significant  design  flexibility  while  yielding  closed  loop  stability. 
All  simulations  were  performed  in  Matlab/Simulink  using  Runge-Kutta,  fourth-order  numerical 
integration. 

Prior  to  giving  the  results  for  the  gyrostat  with  off-axis  rotor,  we  first  show  some  typical  sim¬ 
ulation  results  for  the  oblate  spinup  of  the  four-state  gyrostat  (three  state  design  model  (12.23)) 
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without  damping.  Recall  that  this  model  does  not  have  a  stabilizable  linearization  at  the  origin,  but 
instead  has  qi  {xs)  and  q2  {x^)  states  with  purely  imaginary  eigenvalues,  and  completely  controllable 
u  mode  with  zero  eigenvalue.  Results  are  given  for  two  choices  of  as  discussed  at  the  end  of  the 
previous  section.  In  both  simulations  the  sampling  rate  was  10  Hz^  arbitrarily  chosen  to  be  an 
implementable  number  which  yielded  smooth-looking  trajectories  on  the  time  scale  of  the  problem. 
The  integration  step  size  was  an  order  of  magnitude  smaller,  at  0.01  seconds.  The  initial  condition 
was  (^1,  ^25  =  (sin(10'^),  0,0.2cos(10*^)  -  1),  corresponding  to  the  all-spun  condition  with  an 

initial  cone  angle  of  ten  degrees  and  =  0.2.  In  all  simulations  in  this  section,  we  fix  the  control 
penalty  weight  R  at  the  value  one,  and  for  all  the  oblate  spinup  maneuvers  in  this  section,  we  use 
the  additional  inertia  parameters  ^2  =  -0.3,  iz  =  —0.5.  The  results  for 

0,  0.1) 

are  shown  in  Figure  12.2.  Note  that  the  penalty  value  of  0.1  on  u  was  selected  to  keep  the  control 
input  in  a  neighborhood  of  0.1  or  less,  which  is  known  [23]  to  be  a  reasonable  control  magnitude  for 
oblate  spinup  maneuvers.  As  expected  for  this  choice  of  iJ,  u  is  driven  to  zero,  but  qi  (xs)  and  q2 
{X2)  enter  into  a  limit  cycle  due  to  their  pure  imaginary  axis  eigenvalue  pair.  Note  that  in  order  to 
actually  solve  the  SDRE  for  this  problem,  we  need  solve  only  the  scalar  SDRE 

-h  0.01  =  0 

which  yields  the  linear  control 

u  —  -0,lu 

This  problem  cannot  be  solved  using  a  three-state  parametrization  for  A{x)  because  of  the  lack  of 
detectability. 

The  results  for 

H  =  E2  =  diag{l,  1,  0.1) 

are  shown  in  Figure  12.3.  Note  that  global  observability  is  guaranteed  by  the  global  nonsingularity 
of  H.  The  factorization  used  for  this  simulation  is  the  same  as  (12.34),  except  the  }i2  state  has  been 
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Figure  12.3:  SDRE  State  and  Control  Histories  for  4-State  Gyrostat  {H  —  H2) 
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removed.  Thus, 


A{x)  = 


0  1  -  ^2^1  q2 

i^xi  ~  1  0  —qx 

0  0  0 


(12.56) 


for  this  example.  It  is  easily  verified  that  (12.56)  gives  a  controllable  parametrization  as  long  as  qx 
and  q2  are  not  both  zero.  This  lack  of  controllability  at  the  origin,  however,  prevents  asymptotic 
stability,  as  seen  in  the  figure.  Interestingly,  u  is  not  regulated  much  toward  zero  in  this  case,  for 
which  qx  and  q2  are  being  penalized  ten  times  as  much  as  u.  For  fixed  unit  penalties  on  qx  and  ^2? 
as  the  penalty  on  u  was  increased,  additional  simulations  showed  final  values  of  u  increasingly  closer 
to  zero. 

Finally,  we  compare  these  results  to  the  constant  torque  case  with  u  =  0.02.  The  time  histories 
for  this  case  are  shown  in  Figure  12.4.  Note  that  in  all  cases  the  lack  of  a  stabilizable  linearization 
at  the  origin  prevents  obtaining  asymptotic  closed  loop  stability.  As  a  final  basis  of  comparison,  we 
show  the  xx  time  histories  for  the  three  approaches  in  Figure  12.5.  In  this  and  succeeding  figures, 
xlc  is  the  constant  torque  history,  xlsl  is  the  SDRE  history  for  H  =  Hx,  and  xls2  is  the  SDRE 
history  for  H  —  H2-  As  expected,  none  of  the  Xx  trajectories  asymptotically  approach  the  value  one 
as  desired.  Also,  the  SDRE  case  for  nonsingular  H  has  the  worst  performance  in  terms  of  driving  Xx 
to  the  desired  nonzero  setpoint,  although  it  is  not  much  worse  than  the  others  in  an  absolute  sense. 

We  now  present  simulation  results  for  the  five-state  gyrostat  model,  which  includes  the  off-axis 
rotor  to  provide  damping  and  thus  a  stable  linearization  at  the  origin.  We  give  typical  results 
for  oblate,  prolate,  and  transverse  spinup  maneuvers,  in  that  order.  Recall  that  for  this  system  we 
actually  use  a  four-state  design  model  in  the  SDRE  methods,  so  that  we  use  the  A{x)  parametrization 
given  by  (12.34)  for  all  maneuvers.  We  again  investigate  the  results  of  different  choices  for  ff, 
although  we  restrict  H  to  the  diagonal,  constant  form  given  by  (12.54).  Starting  with  the  oblate 
case,  we  again  show  results  for  simulations  run  at  10  Hz  sampling  rate,  with  inertia  parameters  as 
above.  From  now  on  we  use  Xs  and  X2  when  referring  to  qx  and  q2^  respectively,  so  that  for  these 
simulations  we  start  with  initial  conditions  of  (x’3,  X2,  /i25  *^)  =  (sin(15^),  0,  0,  0.2cos(15®)  —  1), 
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corresponding  to  the  all-spun  condition  with  an  initial  cone  angle  of  15^.  In  computing  the  A(a;) 
factorization,  as  mentioned  before,  we  have  selected  Ip  =  1  so  that  for  the  oblate  case  we  obtain 
0^2  =  1.3  and  ~  1.5.  We  have  also  selected  Is2  =  0.1  so  that  62  =  0.13  and  ^2  =  —1.13  in  (12,34). 
We  show  state  history  results  for 

H  =  Hi  =:  diag{0,  0,  0,  0.1)  (12.57) 

in  Figure  12.6,  and  for 

H  =:H2  =  diag{l,  1,  1,  0.1)  (12.58) 

in  Figure  12.7.  Note  Figure  12.7  has  a  longer  time  scale  to  better  illustrate  convergence  of  u. 

For  purposes  of  comparison,  we  also  show  state  histories  for  a  constant  applied  torque  of  =  0.01 
in  Figure  12.8.  Note  that  even  though  the  u  histories  are  quite  different  for  the  various  simulations 
(in  particular  u  is  driven  to  zero  much  faster  for  H  =  iJi),  there  is  little  difference  between  the 
X2j  and  fi2  histories.  A  comparison  of  the  three  controls  and  xi  histories  are  given  in  Figures 
12.9  and  12.10.  In  the  figures,  as  before,  the  c  suffix  denotes  the  constant  torque  case,  whereas  si 
corresponds  to  SDRE  with  H  =  Hi^  and  s2  corresponds  to  SDRE  with  H  H2*  Figure  12.9  is 
interesting  in  that  the  three  controls  show  significantly  different  natures,  leading  to  the  different  u 
histories  observed,  but  the  other  state  histories  are  quite  similar.  Looking  at  Figure  12.10  we  see 
that  SDRE  with  H  =  H2  does  a  slightly  better  job  than  the  other  methods  of  driving  xi  to  one,  but 
there  is  not  much  difference  between  any  of  the  methods.  Conceptually,  this  makes  sense  since  the 
[H  =  H2)  SDRE  case  is  actively  trying  to  drive  X2  and  x^  to  zero,  whereas  the  {H  =  Hi)  SDRE 
and  constant  torque  cases  ignore  the  other  states  and  simply  work  on  regulating  u. 

Before  studying  the  prolate  case,  we  remark  that  the  results  observed  here  agree  with  the  anal¬ 
ysis  of  Hall  [24]  for  the  oblate  case,  in  that  we  expect  the  small,  constant  torque  to  be  effective  in 
performing  the  oblate  spinup  maneuver,  since  no  open  loop  separatrices  are  crossed  along  the  tra¬ 
jectory.  In  the  prolate  case,  which  we  now  address,  we  do  however  expect  the  small  constant  torque 
solutions  to  show  an  increased  cone  angle,  and  thus  the  SDRE  methods  may  offer  some  advantages. 
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For  the  prolate  simulations,  we  use  the  same  equations  and  factorizations  as  for  the  oblate  case, 
but  we  now  change  our  inertia  parameters  to  ^2  ==  0.5  and  is  =  0.3,  with  ii  =  0.2  as  before.  This 
leads  to  =  0.5  and  as  =  0.7.  By  changing  to  0.26,  we  maintain  the  factorization  parameters 
62  and  d2  the  same  as  in  the  oblate  case.  To  facilitate  comparison  between  cases,  we  start  the 
prolate  simulations  from  the  same  initial  condition  as  wfts  used  in  the  oblate  simulations,  and  we 
again  sample  at  10  Hz,  This  time  we  start  by  examining  the  small  constant  torque  case.  In  Figure 
12.11  we  give  state  time  histories  for  u  =  0.01  until  u  =  0,  and  then  u  =  0  thereafter.  Two  things 
are  immediately  apparent  from  the  figure.  Note  first  of  all  that  changing  to  the  prolate  case  has 
significantly  increased  the  settling  time  of  the  {x2,  xs^  /X2)  states  over  the  oblate  case.  This  is 
because  in  the  oblate  case,  the  eigenvalues  corresponding  to  these  states  in  the  linearization  about 
the  origin  are  A  —  -0.044  ±  1.46j,  —1.042,  respectively,  whereas  in  the  prolate  case  they  become 
A  =  —0.015  ±  0.56j,  —1.1.  Thus,  the  oscillatory  modes  in  the  prolate  case  have  a  much  slower  decay 
rate.  The  second  thing  to  notice  is  the  large  amplitude  oscillations  in  the  X2  and  2^3  states.  Since 
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Figure  12.11:  Prolate  State  Histories  for  5-State  Gyrostat  {u  =  0.01) 

our  control  strategy  is  concerned  only  with  driving  u  to  zero,  these  states  range  far  from  our  actual 
desired  values  of  zero. 

In  Figures  12.12  and  12.13,  we  show  state  histories  for  the  SDRE  regulators  with  H  —  Hi  and 
H  =  H2,  respectively.  We  again  see  the  longer  time  scale  for  the  prolate  case,  but  note  that  the 
SDRE  regulators  do  not  allow  nearly  as  large  oscillations  of  the  X2  and  Xz  states,  as  reflected  in  the 
plots’  smaller  vertical  scale.  This  should  correspond  to  better  performance  of  these  controllers  in 
driving  xi  to  one.  In  Figures  12.14  and  12.15,  we  compare  control  and  xi  time  histories  for  the  three 
methods.  Indeed,  we  see  that  the  SDRE  methods  perform  better  than  the  constant  torque  controller 
in  driving  xi  to  one,  with  the  H  ^  H2  controller  doing  the  best  job.  The  price  to  be  paid  is  seen  in 
Figures  12.13,  in  which  regulation  of  v  is  seen  to  take  much  longer  for  the  H  =  H2  controller  than 
for  the  other  control  strategies,  and  in  Figure  12.14,  in  which  we  see  the  SDRE  H  —  H2  controller 
applies  a  comparatively  large  magnitude  initial  value  of  the  control. 

We  conclude  this  section  with  results  for  the  transverse  spinup  maneuver,  assuming  the  same 
inertia  values  used  for  the  prolate  spacecraft  just  examined.  A  sampling  rate  of  10  iifz  was  again 
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used,  and  the  initial  condition  was  (^3,  0^25  =  (sin(75^),  0,  0,  0.2  cos(75®)  — 1),  corresponding 

to  the  all-spun  condition  at  an  initial  75®  cone  angle.  In  Figures  12.16  and  12.17  we  show  the  state 
time  histories  for  the  SDRE  nonlinear  regulator  with  —  J?  1  and  H  =  H2,  respectively. 

Note  that  for  H  =  Hi,  u  is  driven  to  zero  fairly  quickly  (within  about  fifty  seconds),  but  the 
and  X3  states  have  large  initial  oscillations,  while  for  H  =  H2,  the  initial  oscillations  of  X2  and  0:3 
are  comparatively  smaller  than  for  H  —  Hi,  but  the  x  states  decay  more  slowly,  and  u  takes  much 
longer  to  go  to  zero.  These  behaviors  are  intuitively  satisfying  in  that  they  reflect  either  the  zero 
or  nonzero  penalty  on  integrals  of  deviations  of  the  X2  and  states  in  the  two  cost  functions,  and 
the  resulting  emphasis  on  controlling  either  u  only  {H  =  Hi)  or  a.  weighted  combination  of  all  the 
states  (H  =  H2).  Note  that  it  is  not  apparent  from  Figure  12.17  that  u  actually  does  go  to  zero. 
We  therefore  show  the  continuation  of  the  trajectory  in  this  case  in  Figure  12.18,  and  it  can  be  seen 
that  u  does  indeed  eventually  go  to  zero,  although  it  takes  quite  a  while.  This  settling  time  may 
be  decreased  by  increasing  the  relative  weight  on  i/,  if  desired.  For  comparison  purposes  we  show 
state  histories  for  the  u  =  0.01  constant  torque  transverse  spinup  maneuver  in  Figure  12.19.  Note 
that  the  large  initial  oscillations  in  the  X2  and  X3  states  continue  to  occur  longer  in  the  constant 
torque  case,  and  the  decay  rate  appears  slower  than  in  either  of  the  SDRE  cases.  In  Figures  12.20 
and  12.21  we  show  control  and  xi  histories  for  the  three  methods. 

Note  that,  as  expected,  the  H  =  H2  SDRE  controller  does  the  best  job  of  minimizing  deviations 
of  xi  from  the  desired  value  of  one,  but  uses  a  larger  initial  control  value  to  accomplish  this.  Note 
also  that  the  other  two  methods  allow  Xi  to  go  negative  for  this  simulation,  but  that  does  not  prevent 
effective  regulation  of  the  states  for  these  methods.  It  is  really  only  the  H  —  H2  SDRE  controller 
that  needs  xi  >  0  because  (recall  that)  at  =  0  we  are  no  longer  guaranteed  a  stabilizable  four- 
state  factorization.  Thus,  starting  from  larger  initial  cone  angles  than  75®,  it  may  not  be  possible 
to  use  the  H  =  H2  SDRE  method  since  we  are  likely  to  lose  stabilizability  of  the  factorization  when 
Xi  goes  negative.  This  problem  technically  also  applies  to  the  H  =  Hi  SDRE  controller,  but  can  be 
avoided  by  solving  only  the  scalar  SDRE  involving  if,  which  is  always  controllable  and  observable  if 
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V  is  penalized  appropriately.  For  this  simulation,  however,  the  four-state  SDRE  was  solved  for  the 
H  ^  Hi  case  all  along  the  trajectory  without  any  difficulties. 


12, 7  SDRE  Nonlinear  Hoo  Simulation  Results 

In  this  section  we  briefly  discuss  application  of  the  SDRE  nonlinear  ifoo  control  algorithm  to  the 
gyrostat  example  problem.  Since  we  already  know  that  no  computational  SDRE  method  can  stabilize 
the  original  4-state  gyrostat  model,  our  discussion  here  applies  to  the  same  model  as  used  for  the 
regulator:  the  5-state  gyrostat  model  with  off-axis  rotor  added.  Note  that  we  actually  have  a  4-state 
design  model,  since  we  remove  the  xi  state.  If  we  recall  the  three  additional  assumptions  of  Section 
11.4  necessary  to  extend  the  sampled  data  SDRE  nonlinear  regulator  theory  to  the  nonlinear  Hoo 
case: 


i.  G{x)  is  globally  analytic  with  respect  to  x 

ii.  The  pair  {A{x)^K{x)}  is  globally  stabilizable 

iii.  Stabilizing  solutions  to  (11.51)  exist  everywhere  along  the  trajectory 


and  we  further  invoke  the  assumption  K  <0^  x  sufficient  to  guarantee  satisfaction  of  Assumption 
iii  above,  then  it  is  easily  seen  that  G{x)G^{x)  must  be  of  the  form 

0  0  0  0 

0  0  0  0 

G(x)G'^{x)  =  (12.59) 

0  0  0  0 

0  0  0  gl{x) 

and  additionally  we  must  have 


gl{x)/j^  -  1  <0\/  x 


(12.60) 


Thus,  under  these  assumptions,  the  nonlinear  Hoo  SDRE  is  equivalent  to  the  nonlinear  regulator 
SDRE  with  set  equal  to  B{1  —  gl{x)/'y‘^)B'^  where  0  <  1  —  gl{x)lj'^  <  1,  so  that  at  each 

X  the  nonlinear  Hoo  SDRE  is  equivalent  to  the  regulator  SDRE  with  increased  control  penalty.  If 
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g4{x)  is  assumed  constant,  then  the  point  wise  equivalence  between  the  nonlinear  iJoo  and  scaled 
nonlinear  regulator  SDREs  is  global.  However,  note  that  since  u  =  —R^^B^Px  does  not  see  the 
equivalent  scaling  effect  on  i?,  the  nonlinear  Hoc  control  is  not  the  same  as  the  control  from  the 
scaled  regulator  problem.  If  the  two  were  the  same,  we  would  expect  to  see  reduced  control  effort  for 
the  nonlinear  Hqq  problem,  as  per  the  effect  of  higher  R  in  the  regulator  problem.  However,  since 
the  effect  of  increased  R  in  only  the  SDRE  is  to  make  the  system  appear  less  strongly  controllable 
(it  makes  the  magnitude  of  B  appear  smaller),  we  should  expect  higher  gain  solutions  for  the  SDRE 
nonlinear  if cx>  control  problem.  To  illustrate  this  concept,  we  show  comparative  SDRE  regulator  and 
nonlinear  ifoo  simulation  results  for  the  15^  initial  cone  angle  oblate  case  with  if  —  ifi  in  Figure 
12.22.  Note  that  only  the  control  and  u  state  histories  are  plotted,  and  the  ‘hi’  and  ‘r’  sufBxes  in 
the  figure  correspond  to  the  ifoo  and  regulator  cases,  respectively.  The  other  state  histories  are  not 
plotted  because  they  are  virtually  indistinguishable  for  about  the  first  thirty  seconds,  after  which 
there  is  a  slight  phase  difference  in  the  oscillatory  state  histories.  We  have  assumed 


G{x)  = 


0 

0 


(12.61) 


L.4j 

and  ^4/7^  =  1/2.  The  effect  of  increased  R  in  the  SDRE  is,  as  expected,  higher  initial  control 
gain  and  thus  somewhat  faster  regulation  response.  In  fact,  for  this  case  the  SDRE  solutions  are 
constant  and  are  P44  =  0.1414  and  P44  —  0.1,  respectively,  for  the  nonlinear  Hqo  and  regulator 
cases,  with  all  other  pij  =  0.  This  initial  higher  gain  behavior  in  the  nonlinear  Hqo  setting  carries 
over  to  the  H  —  H2  case  as  well,  as  illustrated  by  Figure  12.23,  in  which  we  have  plotted  the  two 
control  histories  for  this  case.  To  conclude,  we  note  that  the  SDRE  nonlinear  Hoo  theory  for  this 
gyrostat  example  problem,  while  not  adding  much  in  the  way  of  significant  design  flexibility  due  to 
the  rank  deficiency  of  B  and  corresponding  strictness  of  assumptions  made  to  guarantee  existence  of 
SDRE  solutions,  nevertheless  is  seen  to  yield  stabilizing  controllers  under  the  appropriate  additional 
assumptions. 
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Figure  12.23:  SDRE  Nonlinear  Hao  and  Regulator  Controls  for  Oblate  5-State  Gyrostat  {H  =  H2) 
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12,8  Comparison  with  Other  Methods 


In  this  section  we  briefly  examine  the  applicability  of  feedback  linearization  and  recursive  backstep- 
ping  to  the  gyrostat  control  problem,  and  compare  their  usefulness  to  that  of  the  SDRE  methods. 
We  start  with  feedback  linearization.  Recall  from  Section  2.2  that,  if  we  can  solve  the  state  space 
exact  linearization  problem,  then  we  can  turn  the  system  into  the  form 


11  =  Z2 

12  =  ^3 
is  =  ^4 

=u  (12.62) 

which  is  a  controllable  linear  system  that  is  easily  stabilized  to  the  origin.  To  solve  this  problem, 
however,  recall  we  need  an  output  function  for  which  (12.2)  has  relative  degree  four,  and  such  an 
output  exists  if  and  only  if  the  two  controllability-like  conditions  of  Theorem  2.2.3  are  satisfied.  Thus 
As  of  (12.6)  evaluated  at  (a^i,  X2,  p)  —  (1,  0,  0,  1)  must  be  full  rank.  This  matrix  is  only  rank 
one  for  this  problem,  even  if  we  go  to  the  reduced  system  after  eliminating  the  xi  state.  Thus,  the 
state  space  exact  linearization  problem  is  not  solvable  for  the  baseline  gyrostat.  We  are  not  really 
interested  in  an  arbitrary  change  of  coordinates  to  achieve  relative  degree  four  anyway,  since  we 
have  two  known  outputs,  namely  Xi  and  /z,  which  we  desire  to  drive  to  selected  values,  and  they  are 
not  related  by  a  pure  derivative  relationship.  Since  in  feedback  linearization  we  can  independently 
control  only  a  number  of  outputs  strictly  less  than  or  equal  to  the  number  of  inputs,  we  are  thus 
more  interested  in  applying  feedback  linearization  to  the  gyrostat  with  either  X\  or  p  selected  as  the 
output  to  control.  If  we  pick  p  as  our  controlled  output,  and  we  want  to  drive  it  to  one,  this  is  again 
accomplished  by  the  change  of  variables  u  =  p—1  and  regulation  of  u.  Since  we  have  u  =  feedback 
linearization  in  this  case  simply  gives  the  linear  control  law  u  ~  —ku  ~  ~k{p  —  1),  where  A;  >  0 
is  a  user  selectable  constant  that  controls  how  quickly  v  is  regulated.  The  theory  for  input-output 
linearization  then  requires  that  the  zero  dynamics  of  the  remaining  system  be  asymptotically  stable 
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in  order  to  yield  asymptotic  closed  loop  stability,  which  can  be  true  for  this  problem  only  if  we 
consider  the  gyrostat  with  off-axis  rotor  to  provide  damping.  It  is  interesting  to  note  that  this  same 
family  of  controls  is  achieved  through  the  SDRE  nonlinear  regulator  by  taking  ki  =  k2  —  h  =  0 
and  ^4  —  A:  in  (12.54).  Thus,  the  SDRE  framework  allows  us  to  recover  all  controls  that  feedback 
linearization  offers  for  this  choice  of  controlled  output,  and  both  methods  require  the  addition  of  the 
off-axis  rotor  to  give  asymptotic  closed  loop  stability. 

If,  on  the  other  hand,  we  choose  X\  as  the  controlled  ouput,  it  has  relative  degree  two  and  so 
must  be  differentiated  twice  for  the  control  to  appear  in  the  associated  dynamics.  We  have  to  define 
-  1  to  be  able  to  drive  xi  to  one,  and  we  then  get  a  system  of  the  form 

X2  =  cl2{x) 
xs  =  as{x) 

(i  =  u 

xi  =  Zi 
ii  =  Z2 

Z2  =  (^2  -  i3)[ci'i{kxl  -  hxl)  +  40,2^3  -H  (xl  -  xl)u]  (12.63) 

where  the  ai  are  the  right  hand  sides  of  the  original  dynamics  equations  with  Xi  =  xi-\-l  substituted. 
Thus,  to  stabilize  the  linearized  subsystem  of  (12.63),  we  choose  a  control  law  of  the  form 

^  =  T- - - ^{“(*2  -  *3)[ai(*3a;i  -  +  4a2a3  (12.64) 

\}2  ^3)  (^2  ^3) 

where  ^  is  a  constant  gain  vector  which  stabilizes  the  characteristic  equation 

“  93^  —  92^  ~  9i  ~  ^  (12.65) 

We  note  that  this  control  law  is  numerically  ill-conditioned  for  small  X2  and  ^3,  and  undefined  when 
xl  —  x^.  In  addition,  we  still  must  have  stability  of  the  zero  dynamics  to  get  overall  closed  loop 
stability,  and  even  if  we  obtain  stability,  we  still  cannot  use  this  control  law  to  drive  /i  to  one,  since 
the  control  is  already  specfied.  If  we  add  the  off-axis  rotor  equation  to  (12.63),  we  get  a  feedback 
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linearizing  control  of  the  form 


u  — 


_ 1 _ 

(*2  -  *3) (2^2  “  ^3)  +  “2M2a;2 


{-rp{x)+g'^z} 


(12.66) 


where  is  a  complicated  function  of  the  states,  so  that  the  control  is  still  ill-defined  as  the  tra¬ 
jectory  approaches  the  desired  equilibrium  point.  Attempts  to  simulate  this  feedback  linearization 
control  law  exhibited  significant  numerical  instability.  This  control  law,  then,  has  several  unattrac¬ 
tive  features  which  do  not  allow  us  to  accomplish  all  that  we  desire  for  this  problem.  The  conclusion 
to  be  reached  from  all  this  is  that  the  SDRE  nonlinear  regulation  method  offers  everything  that  feed¬ 
back  linearization  offers  for  this  problem,  plus  the  additional  design  flexibility  allowed  by  choosing 
to  penalize  multiple  outputs  and  combinations  of  states  in  the  SDRE  controlled  outputs. 

In  applying  recursive  backstepping  to  the  four-state  gyrostat,  we  see  that  the  system  is  not 
in  the  assumed  ideal  strict-feedback  form,  but  that  by  considering  Xi^  either  X2  or  Xs,  and  /i,  we 
have  a  system  in  strict-feedback  form  plus  a  one-state  zero  dynamics  subsystem.  Thus,  we  consider 
recursive  backstepping  applied  to 


Xi  —  CX2X3 

(12.67) 

X2  =  -  XzpL 

(12.68) 

fl  —  U 

(12.69) 

where  c  —  ^2  ”  ^3  >0?  ^iid  we  treat  3:3  as  a  parameter  with  known  dynamics,  and  we  assume  it  can 
be  measured  perfectly  in  real  time.  To  apply  recursive  backstepping,  we  need  to  select  a  stabilizing 
function,  or  desired  value,  for  X2  in  (12.67),  but  first  we  need  to  shift  the  origin  to  be  our  desired 
final  equilbrium  point.  Thus,  let  zi  =  xi  —1  and  1/  =  1  a.s  before,  to  get 


Zi  —  CX2^3 


±2  =  ^3(^1  +  1)2^3  -  ^3  -  ^3^^ 


Choosing  the  stabilizing  function 


^2  =  ”^3^1 


(12.70) 

(12.71) 

(12.72) 


(12.73) 
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would  give  zi  —  -x^zi  if  achieved  perfectly,  which  is  stable  for  all  xs  and  asymptotically  stable  if 
0^3  i=-  0.  We  thus  have  the  error  function 

62  ~  X2  -ha^3-^l  (12.74) 

so  that  the  dynamics  become 

z\  =  6620^3  -  h:^Z\  =  /i  (12.75) 

62  =  hizi  +  1)X3  ~X3-  X3U  +  0^3/1  +  Zi[iy  -f- 1  -  hizi  H-  l)][e2  -  XsZi]  =  /2  (12.76) 

0  =  u  (12.77) 

Proposing  the  Lyapunov  function  V  =  0.5(zf  -f  e|  -h  t/^)  we  find 

V  =  Zifi  H-  62/2  +  (12.78) 

and  we  see  that,  if  we  have  V  <  0  globally,  we  drive  zi,  62,  and  u  to  zero.  This  is  equivalent  to 
driving  Xi  to  one,  and  also  by  (12.74)  to  driving  X2  to  zero,  which  by  the  constraint  xf-hx^+x^  =  1 
gives  Xs  ~  0,  so  that  our  design  objective  would  be  achieved.  However,  when  =  0,  by  (12.78)  we 
cannot  use  the  control  to  make  V  negative,  and  expansion  of  the  terms  in  (12.78)  shows  that  V  is 
not  negative  definite  for  1/  =  0.  Choosing  to  use  xs  instead  of  x^  in  the  above  development  does 
not  change  this  fact,  so  that  this  recursive  backstepping  approach  will  not  allow  us  to  achieve  our 
design  objectives. 

Alternate  approaches  to  applying  recursive  backstepping  to  this  problem  could  be  tried  along 
the  lines  of  the  approaches  used  in  the  feedback  linearization  discussion  above.  Theorem  2.3.2 
would  allow  us  to  choose  a  control  of  the  form  u  =  where  is  a  globally  positive  definite 

function  of  u,  provided  we  knew  a  Lyapunov  function  for  the  remaining  zero  dynamics  subsystem 
with  guaranteed  negative  derivative.  Without  the  oflf-axis  rotor,  we  know  such  a  Lyapunov  function 
does  not  exist  since  the  system  is  only  stable,  and  not  asymptotically  stable.  We  might  be  able 
to  find  such  a  function  for  the  five-state  gyrostat,  however,  in  which  case  we  recover  the  H  =  Hi 
SDRE  approach  or  equivalently,  the  i^-controlled  variable  feedback  linearization  approach,  if  k{u)  is 
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chosen  a  positive  constant.  In  addition,  both  the  SDRE  and  recursive  backstepping  approaches  can 
generate  nonconstant  positive  definite  simply  by  choosing  the  appropriate  control  in  recursive 
backstepping,  and  by  choosing  nonconstant,  positive  definite  k4{u)  in  the  H  factorization  (12.54) 
for  the  SDRE  nonlinear  regulator. 

Finally,  recursive  backstepping  could  be  applied  to  the  Xi  subsystem  differentiated  twice,  as  in 
the  second  feedback  linearization  approach.  This  approach,  although  it  allows  more  flexibility  than 
feedback  linearization,  is  nevertheless  like  the  feedback  linearization  approach  in  that  it  leads  to  an 
ill-defined  control  when  trajectories  approach  the  origin,  and  also  leaves  v'  uncontrolled,  and  thus  is 
not  particularly  useful. 

As  was  mentioned  in  Chapter  2.3,  recursive  backstepping  is  quite  user-dependent,  and  thus  no 
general  claims  of  its  applicability  or  lack  thereof  to  this  problem  can  be  made.  By  this,  we  mean 
another  designer  might  be  able  to  apply  the  method  to  this  problem  with  more  success  than  we  have. 
What  may  be  concluded,  however,  is  that  in  this  section  we  have  tried  the  most  obvious  applications 
of  the  method,  and  have  again  found  that  SDRE  offers  just  as  much  as  recursive  backstepping  does 
for  these  approaches  in  solving  this  problem, 

12.9  Conclusion 

In  this  chapter  we  have  successfully  applied  the  sampled  data  SDRE  nonlinear  regulation  algorithm 
to  perform  spinup  maneuvers  of  an  axial  gyrostat,  and  we  investigated  the  utility  of  other  control 
algorithms  in  performing  the  same  maneuvers.  First,  we  saw  that  none  of  the  methods  could  asymp¬ 
totically  stabilize  the  system  without  the  addition  of  the  off-axis  damping  rotor,  and  so  our  system 
configuration  was  modified  to  include  its  presence.  For  the  five-state  gyrostat,  the  SDRE  controllers 
were  compared  to  a  constant  torque  control  that  is  commonly  used  to  perform  the  desired  maneu¬ 
vers,  and  significant  improvement  in  controlling  xi  over  the  constant  torque  control  was  achieved 
in  the  prolate  and  transverse  spinup  manuevers.  An  observed  strength  of  the  SDRE  nonlinear  reg¬ 
ulator  is  that  it  offered  significant  design  flexibility,  while  also  stabilizing  the  system  to  the  desired 
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equilibrium  point.  This  flexibility  was  established  by  finding  factorizations  of  the  dynamics  which 
guaranteed  well-posedness  of  the  control  in  a  hemisphere  of  the  momentum  sphere,  and  penalizing 
the  control  and  different  sets  of  states.  By  penalizing  only  u  with  a  constant  weight,  a  stabilizing 
linear  control  law  was  obtained,  and  it  was  shown  that  this  family  of  control  laws  is  the  same  as 
that  obtained  by  applying  feedback  linearization  to  the  gyrostat  problem.  By  allowing  nonconstant 
weights  on  v  only,  an  intuitive  set  of  achievable  controls  from  recursive  backstepping  was  recov¬ 
ered.  The  SDRE  algorithm,  however,  also  allows  the  X2  and  states  to  be  influenced,  in  that  by 
penalizing  these  states,  the  control  can  be  made  to  actively  reduce  their  deviations  from  zero  over 
time.  SDRE  nonlinear  regulation  thus  provides  a  systematic  way  of  achieving  blended  performance 
in  terms  of  driving  both  xi  and  fi  to  the  desired  operating  points,  and  this  is  a  capability  that  none 
of  the  other  methods  offer. 
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XIII.  Conclusions  and  Recommendations  for  Further  Research 


In  this  chapter  we  include  a  brief  summary  of  this  research,  highlighting  the  specific  contributions 
made.  We  draw  some  conclusions  based  on  the  results  obtained,  and  conclude  with  recommendations 
for  further  research. 

13,1  Summary  of  Conducted  Research 

In  this  dissertation,  we  have  conducted  a  fairly  thorough  survey  of  modern  nonlinear  control  design 
techniques,  identifying  the  current  state  of  the  art  in  the  most  common  methods,  and  thus,  high¬ 
lighting  potential  areas  for  research.  Following  this  survey  given  in  Chapter  1,  we  directed  our  focus 
to  the  new  nonlinear  control  design  techniques  based  on  state-dependent  Riccati  equations  (SDREs). 
This  choice  of  focus  was  motivated  by  three  major  factors:  recent  success  in  practical  application  of 
the  SDRE  methods;  a  documented  history  in  the  literature  of  applying  linear  control  techniques  to 
pointwise  linear  factorizations  of  nonlinear  systems;  and  a  significant  lack  of  theoretical  justification 
for  use  of  these  methods,  or  for  logical  choices  of  factorizations.  To  obtain  a  basis  for  comparison 
with  the  SDRE  based  methods,  we  selected  three  common  design  methods,  namely  feedback  lin¬ 
earization,  recursive  backstepping,  and  nonlinear  Hoo  control,  and  we  included  the  basic  theoretical 
components  for  control  design  using  each  technique  in  Chapter  2.  We  have  limited  our  scope  to  the 
case  in  which  full  state  feedback  is  available,  and  we  have  focused  our  attention  mainly  on  achieving 
closed  loop  stability  of  nonlinear  systems,  but  we  have  also  been  interested  in  performance  issues 
using  these  methods.  To  facilitate  stability  analysis,  we  have  included  a  number  of  tools,  including 
the  Principle  of  Stability  in  the  First  Approximation  [68],  Lyapunov  theory  [40],  Lasalle’s  Invariance 
Principle  [44],  and  center  manifold  theory  [10]. 

To  enable  comparison  between  the  four  nonlinear  control  methods  given  above  and  to  illustrate 
promising  areas  of  research,  we  applied  the  methods  to  a  motivational  two-state  example  problem  in 
Chapter  3.  From  this  experience  we  were  able  to  make  several  observations.  It  appears  that  feedback 
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linearization  is  the  most  restrictive  method  in  terms  of  the  conditions  which  a  system  must  satisfy  for 
its  use,  and  we  were  also  able  to  see  that  simply  canceling  nonlinearities,  as  feedback  linearization 
does,  is  not  always  the  best  design  approach.  Next,  we  saw  that  recursive  backstepping  is  an 
effective  tool  for  achieving  stabilization,  but  other  performance  objectives  are  not  easily  incorporated. 
Although  the  example  problem  did  not  illustrate  this,  a  potential  issue  for  recursive  backstepping 
is  getting  systems  into  the  required  strict-feedback  form,  and  what  to  do  when  systems  are  not 
transformable  into  such  a  form.  This  issue  was  indeed  a  factor  for  the  gyrostat  example  problem. 
Nonlinear  Hoo  control  theory,  as  opposed  to  recursive  backstepping,  was  seen  to  provide  a  nice 
general  framework  for  posing  control  problems  which  allow  easy  incorporation  of  performance  as  well 
as  closed  loop  stability  objectives.  While  we  were  able  to  manipulate  a  proposed  polynomial  form 
solution  to  give  a  satisfactory  local  answer  for  the  example  problem  of  Chapter  3,  general  methods  of 
solving  the  resulting  Hamilton-Jacobi  inequalities  (Issacs  equations)  were  nevertheless  needed.  The 
SDRE  method  was  seen  to  offer  promise  in  this  regard;  however,  a  number  of  issues  having  to  do 
with  using  a  storage  function  as  a  Lyapunov  function  needed  to  be  worked  out.  The  SDRE  nonlinear 
regulator  theory  as  presented  by  Cloutier,  et  al  [13]  was  also  promising,  yet  this  method  was  seen  to 
have  the  same  challenge  of  guaranteeing  closed  loop  stability,  and  raised  additional  questions  about 
choices  of  SDC  parametrizations,  and  the  impact  of  controllability /observability  tests  based  on  the 
standard  linear  theory. 

Based  on  the  results  of  Chapter  3  and  a  detailed  study  of  the  existing  theory  for  use  of  SDRE 
methods,  a  number  of  design  issues  were  addressed  in  Chapter  4.  To  enable  subsequent  analysis, 
formulas  for  partial  derivatives  of  vector  matrix  products  with  x  dependency  were  given  in  Sec¬ 
tion  4.1.  The  implications  of  convexity  to  sufficiency  of  local  optimal  solutions  and  global  optima 
were  then  addressed,  and  a  sufficient  condition  for  convexity  of  the  optimal  regulator  cost  function 
was  given,  using  the  above-mentioned  vector  matrix  product  gradient  results.  In  the  next  several 
sections,  issues  concerning  existence  of  optimal  cost  functions  and  their  use  as  system  Lyapunov 
functions  were  clarified.  In  particular,  symmetry  issues  of  the  SDRE  solutions  were  addressed,  and 
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a  simple  form  sufficient  condition  for  satisfying  the  first-order  necessary  conditions  for  optimality  was 
derived.  It  was  seen  that,  although  local  solutions  satisfying  these  first-order  necessary  conditions 
were  obtainable  for  simple  systems,  extending  the  region  of  validity  of  the  results,  especially  for  high 
dimensional  systems,  was  prohibitive  in  terms  of  development  and  implementation  costs.  Thus,  the 
decision  was  made  to  focus  on  suboptimal  solutions,  and  to  seek  means  of  guaranteeing  stability 
other  than  through  the  use  of  optimal  cost  functions  as  Lyapunov  functions.  Also,  analytic  solution 
methods  for  SDREs  were  discarded,  based  on  effort  expended  in  failed  attempts  to  solve  simple 
problems,  in  favor  of  numerical  solutions  using  standard  software.  Finally,  a  comparison  between 
HJIs  and  HJIEs  was  made,  and  a  necessary  condition  for  solvability  of  HJIs  and  HJB  equations  was 
given,  introducing  the  notions  of  nonlinear  controllability  and  stabilizability. 

In  the  next  several  chapters,  we  focused  on  the  need  to  provide  stability  guarantees  for  closed 
loop  systems  using  suboptimal  nonlinear  SDRE  regulators.  In  Chapter  5,  necessary  and  sufficient 
conditions  for  locally  asymptotically  stabilizing  analytic  solutions  for  scalar  analytic  systems  were 
derived  and  presented.  In  Chapter  6,  the  general  nonequivalence  relationship  between  true  nonlinear 
controllability  and  linear  controllability  of  SDC  factorizations  of  nonlinear  systems  was  developed, 
and  the  impact  of  nonlinear  stabilizability  on  successful  application  of  SDRE  methods  was  assessed. 
Particular  cases  where  both  types  of  controllability  hold  were  derived,  and  one  of  these  correspond¬ 
ing  to  full  rank,  constant  B  matrices  motivated  the  global  asymptotic  stability  proof  for  SDRE 
regulators  using  positive  definite  Q  matrices  given  in  Chapter  7.  In  the  next  two  chapters,  sampled 
data  implementations  of  continuous  SDRE  regulators  were  considered,  and  conditions  guaranteeing 
semiglobal  asymptotic  closed  loop  stability  were  presented  and  proven.  Key  elements  of  the  anal¬ 
ysis  in  these  two  chapters  were  state  transition  matrix  representations  for  nonlinear,  discrete  time 
systems;  definition  of  an  appropriate  Lyapunov  function;  and  positive  semidefiniteness  of  the  state 
weighting  matrix  Q  leading  to  invariant  set  theory  in  the  SDRE  framework,  as  needed  for  application 
of  Lasalle’s  Invariance  Principle.  The  theory  of  Chapters  8  and  9  greatly  clarified  a  number  of  issues 
regarding  stability  of  SDRE  regulators  and  appropriate  selection  of  SDC  factorizations.  Then,  in 
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Chapter  10,  a  line  of  inquiry  leading  to  semiglobal  exponential  stability  of  SDRE  regulators  was 
investigated,  showing  the  potential  for  such  a  characteristic  to  exist.  To  conclude  our  theoretical 
development,  in  Chapter  11  we  showed  that  all  the  stability  analysis  performed  for  the  SDRE  reg¬ 
ulator  in  Chapters  5  through  10  could  be  extended  to  the  SDRE  nonlinear  Hoo  setting,  provided 
suitable  additional  assumptions  are  made.  We  also  gave  or  outlined  modified  proofs  for  many  of  the 
stability  theorems  in  the  Hoo  case,  showing  the  impact  of  the  needed  assumptions. 

In  Chapter  12,  a  nontrivial  axial  gyrostat  example  problem  was  investigated,  illustrating  the 
SDRE  stability  and  factorization  theory  developed  in  the  previous  chapters.  Comparison  with  feed¬ 
back  linearization,  recursive  backstepping,  and  constant  torque  controllers  was  also  performed,  and 
the  SDRE  regulator  was  seen  to  allow  great  design  flexibility  and  superior  performance  in  stabiliz¬ 
ing  the  system  to  the  desired  equilibrium  point.  It  was  also  briefly  discussed  how  the  assumptions 
guaranteeing  existence  of  SDRE  solutions  in  the  nonlinear  iJoo  case  limit  the  applicability  of  the 
method  to  the  gyrostat  problem,  in  effect  yielding  solutions  equivalent  to  SDRE  regulators  with 
larger  magnitude  control  penalties.  Since  the  apparent  increase  in  control  penalty  R  was  seen  only 
in  the  SDRE  solution,  these  nonlinear  Hqo  solutions  were  seen  to  produce  larger  initial  magnitude 
controls  than  their  nonlinear  regulator  counterparts. 

13.2  Conclusions 

In  light  of  the  above  discussion,  we  now  conclude  by  identifying  some  of  the  specific  contributions 
this  research  has  made,  and  attempting  to  assess  their  impact.  Through  this  dissertation,  we  have 
made  substantial  progress  in  developing  a  coherent  and  rigorous  theory  for  use  of  the  SDRE  methods. 
We  have  established  sufficient  conditions  for  the  SDRE  regulator  to  be  locally  and  globally  optimal 
and  stable,  including  determining  factorization- dependent  conditions  for  positive  definiteness  of  the 
corresponding  Lyapunov  function.  Making  use  of  the  simplified  necessary  condition  for  optimality  we 
derived,  we  have  proposed  and  validated  a  numerical  local  solution  scheme  to  the  optimal  problem 
for  second-order  systems.  We  have  expressly  chosen  not  to  investigate  globally  optimal  solution 
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algorithms  for  general  systems,  because  we  have  seen  that  the  development  and  implementation  costs 
for  such  an  algorithm  are  quite  likely  to  exceed  the  potential  gain  to  be  had.  We  have  established  a 
necessary  condition  for  solving  general  HJIEs  and  HB J  equations  involving  nonlinear  stabilizability, 
and  we  have  further  made  significant  progress  in  finding  solutions  to  these  equations  using  the  SDRE 
methods.  We  have  examined  in  detail  two  proposed  solution  forms  to  the  nonlinear  Hoo  HJI  via 
SDRE,  concluding  that  symmetric  SDRE  solutions  only  need  be  considered,  in  part  because  we  have 
observed  that  analytical  solutions  will  not  in  general  be  feasible.  Under  conditions  of  symmetry,  we 
have  also  demonstrated  a  clear  relationship  between  the  two  solution  methods,  indicating  one  of  the 
methods  to  be  more  useful,  since  the  other  then  becomes  just  a  special  case  of  the  first,  solvable  only 
for  linear  time  invariant  systems.  We  have  also  investigated  solution  of  strict  HJIs  versus  HJIEs, 
showing  that  solving  a  strict  HJI  is  equivalent  to  solving  an  HJIE  with  a  positive  definite  penalized 
output  norm.  Thus,  solving  strict  HJIs  is  a  simpler  way  to  guarantee  closed  loop  stability,  but  it  is 
in  general  a  harder  problem,  which  may  restrict  solvability  for  a  given  problem. 

In  the  latter  chapters  of  this  dissertation,  we  have  made  significant  strides  in  determining  con¬ 
ditions  which  guarantee  closed  loop  stability  of  both  SDRE  regulators  and  SDRE  nonlinear  Hoo 
controllers,  in  both  continuous  time  and  sampled  data  formats.  For  scalar  analytic  systems,  we 
have  established  necessary  and  sufficient  conditions  for  analytic  locally  stabilizing  solutions,  show¬ 
ing  that  stabilizability /detectability  of  the  regulator  SDC  factorization  is  sufficient,  but  not  at  all 
necessary  for  local  stability  (as  also  observed  earlier  in  the  two-state  example  problem  of  Chapter  3). 
We  examined  true  and  factored  controllability  of  nonlinear  and  SDC  systems,  establishing  a  local 
equivalence  relationship  between  the  two,  and  other  special  cases  when  equivalence  holds,  although 
the  two  concepts  are,  in  general,  different.  Using  this  knowledge,  the  matrix  measure  and  other 
researchers’  work  [49]  on  bounds  of  solutions  to  algebraic  Riccati  equations,  we  were  able  to  prove 
global  asymptotic  stability  of  continuous  time  systems  with  full  rank,  constant  B  matrices,  and  pos¬ 
itive  definite  Q  matrices,  assuming  some  slight  additional  conditions  were  satisfied.  The  sufficiency 
of  having  stabilizable  and  detectable  factorizations  for  well-posedness  of  the  numerically  computed 
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SDRE  regulator  was  established,  based  on  existence  of  stabilizing  solutions  to  the  SDRE,  By  focus¬ 
ing  on  numerical,  stabilizing  solutions  to  AREs,  we  forego  taking  advantage  of  the  observed  ability  of 
SDRE  methods  to  stabilize  systems  locally  that  do  not  possess  stabilizable  and  detectable  factoriza¬ 
tions,  however.  It  was  also  discovered  that  stabilizability /detect ability  of  the  SDC  parametrizations 
gives  analytic  SDRE  solutions,  a  fact  which  was  used  to  help  prove  asymptotic  stability  of  sampled 
data  SDRE  regulators.  The  effect  of  stabilizability  and  detectability  in  factorizations  on  closed  loop 
stability  was  also  identified,  based  on  pointwise  decompositions  of  nonlinear  systems  and  invariant 
set  theory.  In  particular,  we  saw  that  states  in  the  null  space  of  H[x)  do  not  receive  any  control 
action,  so  that  invariant  sets  in  the  null  space  of  H  must  be  asymptotically  stable  in  order  to  obtain 
closed  loop  stability.  Detectability  of  the  factorization  does  not,  in  general,  guarantee  this,  so  that 
additional  analysis  may  be  necessary.  Choosing  H  to  be  globally  nonsingular  so  that  Q  is  positive 
definite  is  then  well  motivated  to  achieve  a  negative  Lyapunov  function  derivative,  but  may  not  lead 
to  a  stable  closed  loop  system  when  the  system  is  not  globally  nonlinearly  stabilizable,  as  examples 
illustrated.  Factorized  stabilizability,  on  the  other  hand,  was  seen  to  guarantee  that  convergent 
closed  loop  solutions  will  not  be  attracted  to  the  uncontrollable  space,  so  that  the  above  analysis 
based  on  observability  concepts  plays  the  determining  role  in  guaranteeing  closed  loop  stability.  We 
then  saw  that,  if  one  can  choose  the  Q  matrix  function  to  give  a  diagonalizable  and  sufficiently  sta¬ 
ble  closed  loop  dynamics  matrix  function,  F,  then  exponential  stability  of  the  sampled  data  SDRE 
regulator  may  be  obtained.  Finally,  by  making  some  additional  assumptions  on  the  disturbance  to 
the  system  dynamics,  we  saw  that  the  above  arguments  can  be  successfully  extended  to  the  SDRE 
nonlinear  Hoo  case. 

We  can  also  draw  several  conclusions  from  application  of  the  SDRE  nonlinear  regulator  to  the 
gyrostat  problem.  We  saw  that  successful  application  of  the  method  hinges  on  being  able  to  find  a 
stabilizable  and  detectable  SDC  factorization  for  the  system,  and  that  finding  such  a  parametriza- 
tion  can  be  a  nontrivial  exercise  for  larger  dimension  systems.  Intuitively  speaking,  attempting  to 
maximize  the  controllable  and  observable  spaces  of  the  factorizations  in  a  linear  sense  is  a  good  rule 
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of  thumb  for  selecting  parametrizations,  since  pointwise  control  effort  can  be  directly  linked  to  these 
issues.  We  also  saw  a  potential  limitation  of  the  numerical  implementation  of  SDRE  methods,  in  that 
they  revert  to  standard  linear  design  techniques  acting  on  the  linearized  dynamics  in  a  neighborhood 
of  the  origin,  and  thus  may  or  may  not  stabilize  systems  that  a  purely  nonlinear  controller  could. 
As  seen  in  Chapter  3,  controls  based  on  analytic  solutions  to  the  SDRE  can  potentially  avoid  this 
limitation.  The  simulation  results  verified  the  Lyapunov  stability  analysis  given  in  this  dissertation 
in  that,  given  nonlinear  controllability,  we  obtained  convergent  SDRE  solutions,  so  that  trajectories 
went  to  the  equilibrium  point  at  the  origin.  This  result  was  expected,  based  on  the  fact  that  we 
had  stabilizable  and  detectable  factorizations,  and  sufficiently  rapid  sampling.  Although  the  effect 
of  various  sampling  rates  was  not  illustrated  for  this  problem,  it  was  observed  in  other  simulations 
conducted  during  this  research  that  decreasing  the  sampling  rate  beyond  a  critical  level  can  indeed 
lead  to  instability.  In  addition  to  providing  the  desired  closed  loop  asymptotic  stability  for  this 
problem,  the  SDRE  method  allowed  significant  design  flexibility  through  the  option  of  penalizing 
various  combinations  of  states.  In  fact,  it  was  seen  that  the  SDRE  technique  could  produce  all  the 
controllers  one  would  reasonably  expect  from  applying  feedback  linearization  and  recursive  back- 
stepping  to  this  problem,  plus  additional  controllers.  This  systematic  design  flexibility  is  therefore 
judged  to  be  one  important  advantage  the  SDRE  method  oflTers.  Finally,  this  flexibility  was  key  to 
the  SDRE  nonlinear  regulator  being  able  to  achieve  better  cone  angle  regulation  performance  than 
the  commonly-used  constant  torque  maneuver. 

In  conclusion,  this  research  has  provided  theoretical  justification  and  demonstration  of  applica¬ 
bility  of  SDRE-based  control  techniques.  Stability  analysis  and  factorization  selection  are  two  major 
areas  where  we  have  contributed,  although  other  contributions  have  also  been  made.  We  have  been 
able  to  make  an  objective  evaluation  and  comparison  of  available  nonlinear  control  methods  with  the 
SDRE  methods,  and  have  seen  that  SDRE  offers  a  number  of  design  advantages,  including  relative 
ease  of  application  if  appropriate  factorizations  are  available. 
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13.3  Recommendations  for  Further  Research 


There  are  a  number  of  promising  areas  for  further  research  which  have  become  obvious  as  this 
research  has  progressed.  We  give  a  brief  list  of  such  below. 

•  Develop  coherent  and  rigorous  theory  for  output  feedback  SDRE  nonlinear  regulation  and  ifoo 
control 

•  Extend  the  theoretical  analyis  of  SDRE  methods  to  include  neutrally  stabilizing  solutions  to 
SDREs,  thereby  not  requiring  stabilizable  and  detectable  factorizations 

•  Fully  develop  the  theory  for  exponential  stability  of  SDRE  controllers 

•  Prove  that  boundedness,  as  opposed  to  convergence,  of  SDRE  solutions  is  sufficient  for  semiglobal 
asymptotic  stability  of  observable  trajectories 

•  Extend  the  global  asymptotic  stability  results  for  full  rank,  constant  B  matrices  to  stability  of 
systems  having  B  matrix  functions  with  some  constant  submatrix 

•  Find  a  tight  bound  on  the  nonoptimality  of  the  SDRE  suboptiraal  nonlinear  regulator 

•  Establish  a  theoretical  proof  of  boundedness  of  SDRE  solutions  given  full  nonlinear  controlla¬ 
bility  of  the  open  loop  system 

•  Analyze  the  global  induced  L2  gain  properties  of  the  SDRE  nonlinear  Hoo  control  algorithm 

•  Treat  the  full-scale  sampled  data  regulator  problem  for  the  equivalent  discrete  time  system,  as 
opposed  to  the  sampled  data  implementation  of  the  continuous  control  law 

It  is  hoped  that  these  areas  of  research  will  be  effectively  pursued,  because  complete  theoretical 
developments  of  the  above  topics  would  greatly  enhance  the  utility  of  the  SDRE  methods. 
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